You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Punith S <pu...@cloudbyte.com> on 2015/02/16 06:51:37 UTC
Query on snapshot and cloning for managed storage
Hi Stackers,
I have been testing out the cloudstack 4.5 RC on vmware setup with
Cloudbyte as a storage backend for supporting managed storage, while taking
a snapshot of a data disk i'm only taking the snapshot from the storage
side only, but on cloning the snapshot of the volume is it enough to just
clone the particular volume on the storage side ? so that the vmdk also be
reproduced again!
on the default primary storage i'm guessing, cloning will be just a copy of
a vmdk file on the same primary datastore!
thanks
--
regards,
punith s
cloudbyte.com
Re: Query on snapshot and cloning for managed storage
Posted by Mike Tutkowski <mi...@solidfire.com>.
Yep, it is unfortunate that XenServer is not as flexible as ESXi is in this
regard. It means we can't use our backend snapshot technologies. We have to
make a new volume and use front-end CPU cycles and network bandwidth to
copy the VDI.
Not an ideal situation, but it's what we can offer in CS 4.6 given the
current state of XenServer.
I have sent an e-mail to XenServer's dev@ mailing list about this, but have
not received a response (I probably sent the message about a month ago).
Perhaps Tim Mackey (who I have CCed here) will be able to help us get this
feature request into them.
It just depends on the SAN as to whether or not that many volumes will be a
problem. It won't be a problem for SolidFire and the SolidFire SAN
de-duplicates blocks inline on a 4-KB boundary, so this extra data written
to the SAN will lead to very little extra data actually being written to
the SAN (and the process of copying from the hypervisor to the SAN will be
much faster than copying from the hypervisor to NFS in most situations).
On Tue, Feb 17, 2015 at 11:09 PM, Punith S <pu...@cloudbyte.com> wrote:
> thanks for the detailed information mike.
>
> since there is a UUID confilct in XenServer we are left with only one
> option, that is to copy the VDI over the network to the new volume created
> during the process of taking volume snapshot.
>
> but my concern is, when an admin takes a recurring snapshots is it
> feasible to create new volumes in the storage backend to all the respective
> snapshots ?
> and also we are not leveraging the cloning ability of our storage
> solutions.!
>
> but on the VMware side, it must be possible to change the datastore
> metadata. hence it will be easy to introduce our storage snapshots meaning
> we don't need to copy the virtual disk(vmfs or vmks) to the new volume
> created like the approach mentioned in using XenServer.
>
> thanks
>
>
> On Mon, Feb 16, 2015 at 11:16 PM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> Hi Punith,
>>
>> I sent this e-mail (below) in a different CloudStack dev@ thread, but it
>> seems they are only interested in RBD in that thread, so I will include
>> that e-mail text in this thread so that we keep track on the mailing list
>> of current issues in XenServer and ESXi with regards to leveraging backend
>> SAN snapshots:
>>
>> *** Beginning of e-mail ***
>>
>> The main requests were around the following features:
>>
>> * The ability to leverage SolidFire snapshots.
>>
>> * The ability to create CloudStack templates from SolidFire snapshots.
>>
>> I had these on my roadmap, but bumped the priority up and began work on
>> them for the CS 4.6 release.
>>
>> During design, I realized there were issues with the way XenServer is
>> architected that prevented me from directly using SolidFire snapshots.
>>
>> I could definitely take a SolidFire snapshot of a SolidFire volume, but
>> this snapshot would not be usable from XenServer if the original volume was
>> still in use.
>>
>> Here is the gist of the problem:
>>
>> When XenServer leverages an iSCSI target such as a SolidFire volume, it
>> applies a clustered files system to it, which they call a storage
>> repository (SR). An SR has an *immutable* UUID associated with it.
>>
>> The virtual volume (which a VM sees as a disk) is represented by a
>> virtual disk image (VDI) in the SR. A VDI also has an *immutable* UUID
>> associated with it.
>>
>> If I take a snapshot (or a clone) of the SolidFire volume and then later
>> try to use that snapshot from XenServer, XenServer complains that the SR on
>> the snapshot has a UUID that conflicts with an existing UUID.
>>
>> In other words, it is not possible to use the original SR and the
>> snapshot of this SR from XenServer at the same time, which is critical in a
>> cloud environment (to enable creating templates from snapshots).
>>
>> The way I have proposed circumventing this issue is not ideal, but
>> technically works (this code is checked into the CS 4.6 branch):
>>
>> When the time comes to take a CloudStack snapshot of a CloudStack volume
>> that is backed by SolidFire storage via the storage plug-in, the plug-in
>> will create a new SolidFire volume with characteristics (size and IOPS)
>> equal to those of the original volume.
>>
>> We then have XenServer attach to this new SolidFire volume, create a
>> *new* SR on it, and then copy the VDI from the source SR to the destination
>> SR (the new SR).
>>
>> This leads to us having a copy of the VDI (a "snapshot" of sorts), but it
>> requires CPU cycles on the compute cluster as well as network bandwidth to
>> write to the SAN (thus it is slower and more resource intensive than a
>> SolidFire snapshot).
>>
>> I spoke with Tim Mackey (who works on XenServer at Citrix) concerning
>> this issue before and during the CloudStack Collaboration Conference in
>> Budapest in November. He agreed that this is a legitimate issue with the
>> way XenServer is designed and could not think of a way (other than what I
>> was doing) to get around it in current versions of XenServer.
>>
>> One thought is to have a feature added to XenServer that enables you to
>> change the UUID of an SR and of a VDI.
>>
>> If I could do that, then I could take a SolidFire snapshot of the
>> SolidFire volume and issue commands to XenServer to have it change the
>> UUIDs of the original SR and the original VDI. I could then recored the
>> necessary UUID info in the CS DB.
>>
>> *** End of e-mail ***
>>
>> Talk to you later,
>> Mike
>>
>> On Sun, Feb 15, 2015 at 11:31 PM, Punith S <pu...@cloudbyte.com>
>> wrote:
>>
>>> thanks mike, i'll go through the classes and i'll try to test it on a
>>> master(4.6) setup for the better understanding.
>>>
>>> On Mon, Feb 16, 2015 at 11:57 AM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> I think it might be easier if you examine my plug-in along with two new
>>>> classes:
>>>>
>>>> StorageSystemSnapshotStrategy
>>>> StorageSystemDataMotionStrategy
>>>>
>>>> Those last two classes are intended for managed storage in general
>>>> (i.e. they are not SolidFire specific).
>>>>
>>>> That code currently only handles XenServer.
>>>>
>>>> I'm currently working on equivalent support in ESXi.
>>>>
>>>>
>>>> On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com> wrote:
>>>>
>>>>> thanks for the info mike.i'll go through your commits of 4.6.
>>>>>
>>>>> the updated plugin i pushed for 4.5 used to only take backend storage
>>>>> snapshot in cloudbyte. but when i'm trying to create a volume out of that
>>>>> snapshot it is going to motion service where it is trying to copy the vmfs
>>>>> or vmdk file but since there is no destination( storage volume cloned out
>>>>> of snapshot) it is failed to create a volume.
>>>>>
>>>>> mike can you please brief me about when we have to make use of
>>>>> canCopy() and copyAsync() interfaces of datastore interfaces.
>>>>>
>>>>> thanks
>>>>>
>>>>> On Mon, Feb 16, 2015 at 11:40 AM, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> I would recommend you take a look at a bunch of my most recent 4.6
>>>>>> commits.
>>>>>>
>>>>>>
>>>>>> On Sunday, February 15, 2015, Mike Tutkowski <
>>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>>
>>>>>>> I am working on creating the necessary infrastructure to support
>>>>>>> managed snapshots on XenServer and ESXi in 4.6 (for block storage, that is).
>>>>>>>
>>>>>>> For example, with XenServer, you cannot simply take
>>>>>>> a backend snapshot of the LUN that contains the SR and VDI.
>>>>>>>
>>>>>>> Technically you can, however, you would find that your snapshot
>>>>>>> cannot be attached to any of the XenServer hosts in the cluster if the
>>>>>>> original LUN you took a snapshot of is still attached in that cluster (due
>>>>>>> to duplicate immutable UUIDs on the SRs and VDIs).
>>>>>>>
>>>>>>> This is an issue I have been discussing with someone who works on
>>>>>>> XenServer.
>>>>>>>
>>>>>>> ESXi is a bit better here since it allows you to re-signature a
>>>>>>> datastore (so taking a backend snapshot should work nicely in 4.6 once I
>>>>>>> finish the code).
>>>>>>>
>>>>>>> If you are talking about NFS, it's probably okay if you just take a
>>>>>>> backend snapshot of the VMDK file in question.
>>>>>>>
>>>>>>> On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Stackers,
>>>>>>>>
>>>>>>>> I have been testing out the cloudstack 4.5 RC on vmware setup with
>>>>>>>> Cloudbyte as a storage backend for supporting managed storage, while taking
>>>>>>>> a snapshot of a data disk i'm only taking the snapshot from the storage
>>>>>>>> side only, but on cloning the snapshot of the volume is it enough to just
>>>>>>>> clone the particular volume on the storage side ? so that the vmdk also be
>>>>>>>> reproduced again!
>>>>>>>>
>>>>>>>> on the default primary storage i'm guessing, cloning will be just a
>>>>>>>> copy of a vmdk file on the same primary datastore!
>>>>>>>>
>>>>>>>> thanks
>>>>>>>> --
>>>>>>>> regards,
>>>>>>>>
>>>>>>>> punith s
>>>>>>>> cloudbyte.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Mike Tutkowski*
>>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>>> e: mike.tutkowski@solidfire.com
>>>>>>> o: 303.746.7302
>>>>>>> Advancing the way the world uses the cloud
>>>>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud
>>>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> regards,
>>>>>
>>>>> punith s
>>>>> cloudbyte.com
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud
>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>
>>>>
>>>
>>>
>>> --
>>> regards,
>>>
>>> punith s
>>> cloudbyte.com
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud
>> <http://solidfire.com/solution/overview/?video=play>*™*
>>
>
>
>
> --
> regards,
>
> punith s
> cloudbyte.com
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*
Re: Query on snapshot and cloning for managed storage
Posted by Punith S <pu...@cloudbyte.com>.
thanks for the detailed information mike.
since there is a UUID confilct in XenServer we are left with only one
option, that is to copy the VDI over the network to the new volume created
during the process of taking volume snapshot.
but my concern is, when an admin takes a recurring snapshots is it feasible
to create new volumes in the storage backend to all the respective
snapshots ?
and also we are not leveraging the cloning ability of our storage
solutions.!
but on the VMware side, it must be possible to change the datastore
metadata. hence it will be easy to introduce our storage snapshots meaning
we don't need to copy the virtual disk(vmfs or vmks) to the new volume
created like the approach mentioned in using XenServer.
thanks
On Mon, Feb 16, 2015 at 11:16 PM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> Hi Punith,
>
> I sent this e-mail (below) in a different CloudStack dev@ thread, but it
> seems they are only interested in RBD in that thread, so I will include
> that e-mail text in this thread so that we keep track on the mailing list
> of current issues in XenServer and ESXi with regards to leveraging backend
> SAN snapshots:
>
> *** Beginning of e-mail ***
>
> The main requests were around the following features:
>
> * The ability to leverage SolidFire snapshots.
>
> * The ability to create CloudStack templates from SolidFire snapshots.
>
> I had these on my roadmap, but bumped the priority up and began work on
> them for the CS 4.6 release.
>
> During design, I realized there were issues with the way XenServer is
> architected that prevented me from directly using SolidFire snapshots.
>
> I could definitely take a SolidFire snapshot of a SolidFire volume, but
> this snapshot would not be usable from XenServer if the original volume was
> still in use.
>
> Here is the gist of the problem:
>
> When XenServer leverages an iSCSI target such as a SolidFire volume, it
> applies a clustered files system to it, which they call a storage
> repository (SR). An SR has an *immutable* UUID associated with it.
>
> The virtual volume (which a VM sees as a disk) is represented by a virtual
> disk image (VDI) in the SR. A VDI also has an *immutable* UUID associated
> with it.
>
> If I take a snapshot (or a clone) of the SolidFire volume and then later
> try to use that snapshot from XenServer, XenServer complains that the SR on
> the snapshot has a UUID that conflicts with an existing UUID.
>
> In other words, it is not possible to use the original SR and the snapshot
> of this SR from XenServer at the same time, which is critical in a cloud
> environment (to enable creating templates from snapshots).
>
> The way I have proposed circumventing this issue is not ideal, but
> technically works (this code is checked into the CS 4.6 branch):
>
> When the time comes to take a CloudStack snapshot of a CloudStack volume
> that is backed by SolidFire storage via the storage plug-in, the plug-in
> will create a new SolidFire volume with characteristics (size and IOPS)
> equal to those of the original volume.
>
> We then have XenServer attach to this new SolidFire volume, create a *new*
> SR on it, and then copy the VDI from the source SR to the destination SR
> (the new SR).
>
> This leads to us having a copy of the VDI (a "snapshot" of sorts), but it
> requires CPU cycles on the compute cluster as well as network bandwidth to
> write to the SAN (thus it is slower and more resource intensive than a
> SolidFire snapshot).
>
> I spoke with Tim Mackey (who works on XenServer at Citrix) concerning this
> issue before and during the CloudStack Collaboration Conference in Budapest
> in November. He agreed that this is a legitimate issue with the way
> XenServer is designed and could not think of a way (other than what I was
> doing) to get around it in current versions of XenServer.
>
> One thought is to have a feature added to XenServer that enables you to
> change the UUID of an SR and of a VDI.
>
> If I could do that, then I could take a SolidFire snapshot of the
> SolidFire volume and issue commands to XenServer to have it change the
> UUIDs of the original SR and the original VDI. I could then recored the
> necessary UUID info in the CS DB.
>
> *** End of e-mail ***
>
> Talk to you later,
> Mike
>
> On Sun, Feb 15, 2015 at 11:31 PM, Punith S <pu...@cloudbyte.com> wrote:
>
>> thanks mike, i'll go through the classes and i'll try to test it on a
>> master(4.6) setup for the better understanding.
>>
>> On Mon, Feb 16, 2015 at 11:57 AM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> I think it might be easier if you examine my plug-in along with two new
>>> classes:
>>>
>>> StorageSystemSnapshotStrategy
>>> StorageSystemDataMotionStrategy
>>>
>>> Those last two classes are intended for managed storage in general (i.e.
>>> they are not SolidFire specific).
>>>
>>> That code currently only handles XenServer.
>>>
>>> I'm currently working on equivalent support in ESXi.
>>>
>>>
>>> On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com> wrote:
>>>
>>>> thanks for the info mike.i'll go through your commits of 4.6.
>>>>
>>>> the updated plugin i pushed for 4.5 used to only take backend storage
>>>> snapshot in cloudbyte. but when i'm trying to create a volume out of that
>>>> snapshot it is going to motion service where it is trying to copy the vmfs
>>>> or vmdk file but since there is no destination( storage volume cloned out
>>>> of snapshot) it is failed to create a volume.
>>>>
>>>> mike can you please brief me about when we have to make use of
>>>> canCopy() and copyAsync() interfaces of datastore interfaces.
>>>>
>>>> thanks
>>>>
>>>> On Mon, Feb 16, 2015 at 11:40 AM, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> I would recommend you take a look at a bunch of my most recent 4.6
>>>>> commits.
>>>>>
>>>>>
>>>>> On Sunday, February 15, 2015, Mike Tutkowski <
>>>>> mike.tutkowski@solidfire.com> wrote:
>>>>>
>>>>>> I am working on creating the necessary infrastructure to support
>>>>>> managed snapshots on XenServer and ESXi in 4.6 (for block storage, that is).
>>>>>>
>>>>>> For example, with XenServer, you cannot simply take
>>>>>> a backend snapshot of the LUN that contains the SR and VDI.
>>>>>>
>>>>>> Technically you can, however, you would find that your snapshot
>>>>>> cannot be attached to any of the XenServer hosts in the cluster if the
>>>>>> original LUN you took a snapshot of is still attached in that cluster (due
>>>>>> to duplicate immutable UUIDs on the SRs and VDIs).
>>>>>>
>>>>>> This is an issue I have been discussing with someone who works on
>>>>>> XenServer.
>>>>>>
>>>>>> ESXi is a bit better here since it allows you to re-signature a
>>>>>> datastore (so taking a backend snapshot should work nicely in 4.6 once I
>>>>>> finish the code).
>>>>>>
>>>>>> If you are talking about NFS, it's probably okay if you just take a
>>>>>> backend snapshot of the VMDK file in question.
>>>>>>
>>>>>> On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Stackers,
>>>>>>>
>>>>>>> I have been testing out the cloudstack 4.5 RC on vmware setup with
>>>>>>> Cloudbyte as a storage backend for supporting managed storage, while taking
>>>>>>> a snapshot of a data disk i'm only taking the snapshot from the storage
>>>>>>> side only, but on cloning the snapshot of the volume is it enough to just
>>>>>>> clone the particular volume on the storage side ? so that the vmdk also be
>>>>>>> reproduced again!
>>>>>>>
>>>>>>> on the default primary storage i'm guessing, cloning will be just a
>>>>>>> copy of a vmdk file on the same primary datastore!
>>>>>>>
>>>>>>> thanks
>>>>>>> --
>>>>>>> regards,
>>>>>>>
>>>>>>> punith s
>>>>>>> cloudbyte.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Mike Tutkowski*
>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>>> e: mike.tutkowski@solidfire.com
>>>>>> o: 303.746.7302
>>>>>> Advancing the way the world uses the cloud
>>>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud
>>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> regards,
>>>>
>>>> punith s
>>>> cloudbyte.com
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud
>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>
>>>
>>
>>
>> --
>> regards,
>>
>> punith s
>> cloudbyte.com
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud
> <http://solidfire.com/solution/overview/?video=play>*™*
>
--
regards,
punith s
cloudbyte.com
Re: Query on snapshot and cloning for managed storage
Posted by Mike Tutkowski <mi...@solidfire.com>.
Hi Punith,
I sent this e-mail (below) in a different CloudStack dev@ thread, but it
seems they are only interested in RBD in that thread, so I will include
that e-mail text in this thread so that we keep track on the mailing list
of current issues in XenServer and ESXi with regards to leveraging backend
SAN snapshots:
*** Beginning of e-mail ***
The main requests were around the following features:
* The ability to leverage SolidFire snapshots.
* The ability to create CloudStack templates from SolidFire snapshots.
I had these on my roadmap, but bumped the priority up and began work on
them for the CS 4.6 release.
During design, I realized there were issues with the way XenServer is
architected that prevented me from directly using SolidFire snapshots.
I could definitely take a SolidFire snapshot of a SolidFire volume, but
this snapshot would not be usable from XenServer if the original volume was
still in use.
Here is the gist of the problem:
When XenServer leverages an iSCSI target such as a SolidFire volume, it
applies a clustered files system to it, which they call a storage
repository (SR). An SR has an *immutable* UUID associated with it.
The virtual volume (which a VM sees as a disk) is represented by a virtual
disk image (VDI) in the SR. A VDI also has an *immutable* UUID associated
with it.
If I take a snapshot (or a clone) of the SolidFire volume and then later
try to use that snapshot from XenServer, XenServer complains that the SR on
the snapshot has a UUID that conflicts with an existing UUID.
In other words, it is not possible to use the original SR and the snapshot
of this SR from XenServer at the same time, which is critical in a cloud
environment (to enable creating templates from snapshots).
The way I have proposed circumventing this issue is not ideal, but
technically works (this code is checked into the CS 4.6 branch):
When the time comes to take a CloudStack snapshot of a CloudStack volume
that is backed by SolidFire storage via the storage plug-in, the plug-in
will create a new SolidFire volume with characteristics (size and IOPS)
equal to those of the original volume.
We then have XenServer attach to this new SolidFire volume, create a *new*
SR on it, and then copy the VDI from the source SR to the destination SR
(the new SR).
This leads to us having a copy of the VDI (a "snapshot" of sorts), but it
requires CPU cycles on the compute cluster as well as network bandwidth to
write to the SAN (thus it is slower and more resource intensive than a
SolidFire snapshot).
I spoke with Tim Mackey (who works on XenServer at Citrix) concerning this
issue before and during the CloudStack Collaboration Conference in Budapest
in November. He agreed that this is a legitimate issue with the way
XenServer is designed and could not think of a way (other than what I was
doing) to get around it in current versions of XenServer.
One thought is to have a feature added to XenServer that enables you to
change the UUID of an SR and of a VDI.
If I could do that, then I could take a SolidFire snapshot of the SolidFire
volume and issue commands to XenServer to have it change the UUIDs of the
original SR and the original VDI. I could then recored the necessary UUID
info in the CS DB.
*** End of e-mail ***
Talk to you later,
Mike
On Sun, Feb 15, 2015 at 11:31 PM, Punith S <pu...@cloudbyte.com> wrote:
> thanks mike, i'll go through the classes and i'll try to test it on a
> master(4.6) setup for the better understanding.
>
> On Mon, Feb 16, 2015 at 11:57 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com> wrote:
>
>> I think it might be easier if you examine my plug-in along with two new
>> classes:
>>
>> StorageSystemSnapshotStrategy
>> StorageSystemDataMotionStrategy
>>
>> Those last two classes are intended for managed storage in general (i.e.
>> they are not SolidFire specific).
>>
>> That code currently only handles XenServer.
>>
>> I'm currently working on equivalent support in ESXi.
>>
>>
>> On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com> wrote:
>>
>>> thanks for the info mike.i'll go through your commits of 4.6.
>>>
>>> the updated plugin i pushed for 4.5 used to only take backend storage
>>> snapshot in cloudbyte. but when i'm trying to create a volume out of that
>>> snapshot it is going to motion service where it is trying to copy the vmfs
>>> or vmdk file but since there is no destination( storage volume cloned out
>>> of snapshot) it is failed to create a volume.
>>>
>>> mike can you please brief me about when we have to make use of canCopy()
>>> and copyAsync() interfaces of datastore interfaces.
>>>
>>> thanks
>>>
>>> On Mon, Feb 16, 2015 at 11:40 AM, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> I would recommend you take a look at a bunch of my most recent 4.6
>>>> commits.
>>>>
>>>>
>>>> On Sunday, February 15, 2015, Mike Tutkowski <
>>>> mike.tutkowski@solidfire.com> wrote:
>>>>
>>>>> I am working on creating the necessary infrastructure to support
>>>>> managed snapshots on XenServer and ESXi in 4.6 (for block storage, that is).
>>>>>
>>>>> For example, with XenServer, you cannot simply take a backend snapshot
>>>>> of the LUN that contains the SR and VDI.
>>>>>
>>>>> Technically you can, however, you would find that your snapshot cannot
>>>>> be attached to any of the XenServer hosts in the cluster if the original
>>>>> LUN you took a snapshot of is still attached in that cluster (due to
>>>>> duplicate immutable UUIDs on the SRs and VDIs).
>>>>>
>>>>> This is an issue I have been discussing with someone who works on
>>>>> XenServer.
>>>>>
>>>>> ESXi is a bit better here since it allows you to re-signature a
>>>>> datastore (so taking a backend snapshot should work nicely in 4.6 once I
>>>>> finish the code).
>>>>>
>>>>> If you are talking about NFS, it's probably okay if you just take a
>>>>> backend snapshot of the VMDK file in question.
>>>>>
>>>>> On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com> wrote:
>>>>>
>>>>>> Hi Stackers,
>>>>>>
>>>>>> I have been testing out the cloudstack 4.5 RC on vmware setup with
>>>>>> Cloudbyte as a storage backend for supporting managed storage, while taking
>>>>>> a snapshot of a data disk i'm only taking the snapshot from the storage
>>>>>> side only, but on cloning the snapshot of the volume is it enough to just
>>>>>> clone the particular volume on the storage side ? so that the vmdk also be
>>>>>> reproduced again!
>>>>>>
>>>>>> on the default primary storage i'm guessing, cloning will be just a
>>>>>> copy of a vmdk file on the same primary datastore!
>>>>>>
>>>>>> thanks
>>>>>> --
>>>>>> regards,
>>>>>>
>>>>>> punith s
>>>>>> cloudbyte.com
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Mike Tutkowski*
>>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>>> e: mike.tutkowski@solidfire.com
>>>>> o: 303.746.7302
>>>>> Advancing the way the world uses the cloud
>>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>>
>>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud
>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>
>>>>
>>>
>>>
>>> --
>>> regards,
>>>
>>> punith s
>>> cloudbyte.com
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud
>> <http://solidfire.com/solution/overview/?video=play>*™*
>>
>>
>
>
> --
> regards,
>
> punith s
> cloudbyte.com
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*
Re: Query on snapshot and cloning for managed storage
Posted by Punith S <pu...@cloudbyte.com>.
thanks mike, i'll go through the classes and i'll try to test it on a
master(4.6) setup for the better understanding.
On Mon, Feb 16, 2015 at 11:57 AM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> I think it might be easier if you examine my plug-in along with two new
> classes:
>
> StorageSystemSnapshotStrategy
> StorageSystemDataMotionStrategy
>
> Those last two classes are intended for managed storage in general (i.e.
> they are not SolidFire specific).
>
> That code currently only handles XenServer.
>
> I'm currently working on equivalent support in ESXi.
>
>
> On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com> wrote:
>
>> thanks for the info mike.i'll go through your commits of 4.6.
>>
>> the updated plugin i pushed for 4.5 used to only take backend storage
>> snapshot in cloudbyte. but when i'm trying to create a volume out of that
>> snapshot it is going to motion service where it is trying to copy the vmfs
>> or vmdk file but since there is no destination( storage volume cloned out
>> of snapshot) it is failed to create a volume.
>>
>> mike can you please brief me about when we have to make use of canCopy()
>> and copyAsync() interfaces of datastore interfaces.
>>
>> thanks
>>
>> On Mon, Feb 16, 2015 at 11:40 AM, Mike Tutkowski <
>> mike.tutkowski@solidfire.com> wrote:
>>
>>> I would recommend you take a look at a bunch of my most recent 4.6
>>> commits.
>>>
>>>
>>> On Sunday, February 15, 2015, Mike Tutkowski <
>>> mike.tutkowski@solidfire.com> wrote:
>>>
>>>> I am working on creating the necessary infrastructure to support
>>>> managed snapshots on XenServer and ESXi in 4.6 (for block storage, that is).
>>>>
>>>> For example, with XenServer, you cannot simply take a backend snapshot
>>>> of the LUN that contains the SR and VDI.
>>>>
>>>> Technically you can, however, you would find that your snapshot cannot
>>>> be attached to any of the XenServer hosts in the cluster if the original
>>>> LUN you took a snapshot of is still attached in that cluster (due to
>>>> duplicate immutable UUIDs on the SRs and VDIs).
>>>>
>>>> This is an issue I have been discussing with someone who works on
>>>> XenServer.
>>>>
>>>> ESXi is a bit better here since it allows you to re-signature a
>>>> datastore (so taking a backend snapshot should work nicely in 4.6 once I
>>>> finish the code).
>>>>
>>>> If you are talking about NFS, it's probably okay if you just take a
>>>> backend snapshot of the VMDK file in question.
>>>>
>>>> On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com> wrote:
>>>>
>>>>> Hi Stackers,
>>>>>
>>>>> I have been testing out the cloudstack 4.5 RC on vmware setup with
>>>>> Cloudbyte as a storage backend for supporting managed storage, while taking
>>>>> a snapshot of a data disk i'm only taking the snapshot from the storage
>>>>> side only, but on cloning the snapshot of the volume is it enough to just
>>>>> clone the particular volume on the storage side ? so that the vmdk also be
>>>>> reproduced again!
>>>>>
>>>>> on the default primary storage i'm guessing, cloning will be just a
>>>>> copy of a vmdk file on the same primary datastore!
>>>>>
>>>>> thanks
>>>>> --
>>>>> regards,
>>>>>
>>>>> punith s
>>>>> cloudbyte.com
>>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkowski@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud
>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>
>>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud
>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>
>>>
>>
>>
>> --
>> regards,
>>
>> punith s
>> cloudbyte.com
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud
> <http://solidfire.com/solution/overview/?video=play>*™*
>
>
--
regards,
punith s
cloudbyte.com
Re: Query on snapshot and cloning for managed storage
Posted by Mike Tutkowski <mi...@solidfire.com>.
I think it might be easier if you examine my plug-in along with two new
classes:
StorageSystemSnapshotStrategy
StorageSystemDataMotionStrategy
Those last two classes are intended for managed storage in general (i.e.
they are not SolidFire specific).
That code currently only handles XenServer.
I'm currently working on equivalent support in ESXi.
On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com> wrote:
> thanks for the info mike.i'll go through your commits of 4.6.
>
> the updated plugin i pushed for 4.5 used to only take backend storage
> snapshot in cloudbyte. but when i'm trying to create a volume out of that
> snapshot it is going to motion service where it is trying to copy the vmfs
> or vmdk file but since there is no destination( storage volume cloned out
> of snapshot) it is failed to create a volume.
>
> mike can you please brief me about when we have to make use of canCopy()
> and copyAsync() interfaces of datastore interfaces.
>
> thanks
>
> On Mon, Feb 16, 2015 at 11:40 AM, Mike Tutkowski <
> mike.tutkowski@solidfire.com
> <javascript:_e(%7B%7D,'cvml','mike.tutkowski@solidfire.com');>> wrote:
>
>> I would recommend you take a look at a bunch of my most recent 4.6
>> commits.
>>
>>
>> On Sunday, February 15, 2015, Mike Tutkowski <
>> mike.tutkowski@solidfire.com
>> <javascript:_e(%7B%7D,'cvml','mike.tutkowski@solidfire.com');>> wrote:
>>
>>> I am working on creating the necessary infrastructure to support managed
>>> snapshots on XenServer and ESXi in 4.6 (for block storage, that is).
>>>
>>> For example, with XenServer, you cannot simply take a backend snapshot
>>> of the LUN that contains the SR and VDI.
>>>
>>> Technically you can, however, you would find that your snapshot cannot
>>> be attached to any of the XenServer hosts in the cluster if the original
>>> LUN you took a snapshot of is still attached in that cluster (due to
>>> duplicate immutable UUIDs on the SRs and VDIs).
>>>
>>> This is an issue I have been discussing with someone who works on
>>> XenServer.
>>>
>>> ESXi is a bit better here since it allows you to re-signature a
>>> datastore (so taking a backend snapshot should work nicely in 4.6 once I
>>> finish the code).
>>>
>>> If you are talking about NFS, it's probably okay if you just take a
>>> backend snapshot of the VMDK file in question.
>>>
>>> On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com> wrote:
>>>
>>>> Hi Stackers,
>>>>
>>>> I have been testing out the cloudstack 4.5 RC on vmware setup with
>>>> Cloudbyte as a storage backend for supporting managed storage, while taking
>>>> a snapshot of a data disk i'm only taking the snapshot from the storage
>>>> side only, but on cloning the snapshot of the volume is it enough to just
>>>> clone the particular volume on the storage side ? so that the vmdk also be
>>>> reproduced again!
>>>>
>>>> on the default primary storage i'm guessing, cloning will be just a
>>>> copy of a vmdk file on the same primary datastore!
>>>>
>>>> thanks
>>>> --
>>>> regards,
>>>>
>>>> punith s
>>>> cloudbyte.com
>>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkowski@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud
>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>
>>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> <javascript:_e(%7B%7D,'cvml','mike.tutkowski@solidfire.com');>
>> o: 303.746.7302
>> Advancing the way the world uses the cloud
>> <http://solidfire.com/solution/overview/?video=play>*™*
>>
>>
>
>
> --
> regards,
>
> punith s
> cloudbyte.com
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*
Re: Query on snapshot and cloning for managed storage
Posted by Punith S <pu...@cloudbyte.com>.
thanks for the info mike.i'll go through your commits of 4.6.
the updated plugin i pushed for 4.5 used to only take backend storage
snapshot in cloudbyte. but when i'm trying to create a volume out of that
snapshot it is going to motion service where it is trying to copy the vmfs
or vmdk file but since there is no destination( storage volume cloned out
of snapshot) it is failed to create a volume.
mike can you please brief me about when we have to make use of canCopy()
and copyAsync() interfaces of datastore interfaces.
thanks
On Mon, Feb 16, 2015 at 11:40 AM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> I would recommend you take a look at a bunch of my most recent 4.6
> commits.
>
>
> On Sunday, February 15, 2015, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
>> I am working on creating the necessary infrastructure to support managed
>> snapshots on XenServer and ESXi in 4.6 (for block storage, that is).
>>
>> For example, with XenServer, you cannot simply take a backend snapshot of
>> the LUN that contains the SR and VDI.
>>
>> Technically you can, however, you would find that your snapshot cannot be
>> attached to any of the XenServer hosts in the cluster if the original LUN
>> you took a snapshot of is still attached in that cluster (due to
>> duplicate immutable UUIDs on the SRs and VDIs).
>>
>> This is an issue I have been discussing with someone who works on
>> XenServer.
>>
>> ESXi is a bit better here since it allows you to re-signature a datastore
>> (so taking a backend snapshot should work nicely in 4.6 once I finish the
>> code).
>>
>> If you are talking about NFS, it's probably okay if you just take a
>> backend snapshot of the VMDK file in question.
>>
>> On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com> wrote:
>>
>>> Hi Stackers,
>>>
>>> I have been testing out the cloudstack 4.5 RC on vmware setup with
>>> Cloudbyte as a storage backend for supporting managed storage, while taking
>>> a snapshot of a data disk i'm only taking the snapshot from the storage
>>> side only, but on cloning the snapshot of the volume is it enough to just
>>> clone the particular volume on the storage side ? so that the vmdk also be
>>> reproduced again!
>>>
>>> on the default primary storage i'm guessing, cloning will be just a copy
>>> of a vmdk file on the same primary datastore!
>>>
>>> thanks
>>> --
>>> regards,
>>>
>>> punith s
>>> cloudbyte.com
>>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkowski@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud
>> <http://solidfire.com/solution/overview/?video=play>*™*
>>
>>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud
> <http://solidfire.com/solution/overview/?video=play>*™*
>
>
--
regards,
punith s
cloudbyte.com
Re: Query on snapshot and cloning for managed storage
Posted by Mike Tutkowski <mi...@solidfire.com>.
I would recommend you take a look at a bunch of my most recent 4.6 commits.
On Sunday, February 15, 2015, Mike Tutkowski <mi...@solidfire.com>
wrote:
> I am working on creating the necessary infrastructure to support managed
> snapshots on XenServer and ESXi in 4.6 (for block storage, that is).
>
> For example, with XenServer, you cannot simply take a backend snapshot of
> the LUN that contains the SR and VDI.
>
> Technically you can, however, you would find that your snapshot cannot be
> attached to any of the XenServer hosts in the cluster if the original LUN
> you took a snapshot of is still attached in that cluster (due to
> duplicate immutable UUIDs on the SRs and VDIs).
>
> This is an issue I have been discussing with someone who works on
> XenServer.
>
> ESXi is a bit better here since it allows you to re-signature a datastore
> (so taking a backend snapshot should work nicely in 4.6 once I finish the
> code).
>
> If you are talking about NFS, it's probably okay if you just take a
> backend snapshot of the VMDK file in question.
>
> On Sunday, February 15, 2015, Punith S <punith.s@cloudbyte.com
> <javascript:_e(%7B%7D,'cvml','punith.s@cloudbyte.com');>> wrote:
>
>> Hi Stackers,
>>
>> I have been testing out the cloudstack 4.5 RC on vmware setup with
>> Cloudbyte as a storage backend for supporting managed storage, while taking
>> a snapshot of a data disk i'm only taking the snapshot from the storage
>> side only, but on cloning the snapshot of the volume is it enough to just
>> clone the particular volume on the storage side ? so that the vmdk also be
>> reproduced again!
>>
>> on the default primary storage i'm guessing, cloning will be just a copy
>> of a vmdk file on the same primary datastore!
>>
>> thanks
>> --
>> regards,
>>
>> punith s
>> cloudbyte.com
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> <javascript:_e(%7B%7D,'cvml','mike.tutkowski@solidfire.com');>
> o: 303.746.7302
> Advancing the way the world uses the cloud
> <http://solidfire.com/solution/overview/?video=play>*™*
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*
Re: Query on snapshot and cloning for managed storage
Posted by Mike Tutkowski <mi...@solidfire.com>.
I am working on creating the necessary infrastructure to support managed
snapshots on XenServer and ESXi in 4.6 (for block storage, that is).
For example, with XenServer, you cannot simply take a backend snapshot of
the LUN that contains the SR and VDI.
Technically you can, however, you would find that your snapshot cannot be
attached to any of the XenServer hosts in the cluster if the original LUN
you took a snapshot of is still attached in that cluster (due to
duplicate immutable UUIDs on the SRs and VDIs).
This is an issue I have been discussing with someone who works on XenServer.
ESXi is a bit better here since it allows you to re-signature a datastore
(so taking a backend snapshot should work nicely in 4.6 once I finish the
code).
If you are talking about NFS, it's probably okay if you just take a backend
snapshot of the VMDK file in question.
On Sunday, February 15, 2015, Punith S <pu...@cloudbyte.com> wrote:
> Hi Stackers,
>
> I have been testing out the cloudstack 4.5 RC on vmware setup with
> Cloudbyte as a storage backend for supporting managed storage, while taking
> a snapshot of a data disk i'm only taking the snapshot from the storage
> side only, but on cloning the snapshot of the volume is it enough to just
> clone the particular volume on the storage side ? so that the vmdk also be
> reproduced again!
>
> on the default primary storage i'm guessing, cloning will be just a copy
> of a vmdk file on the same primary datastore!
>
> thanks
> --
> regards,
>
> punith s
> cloudbyte.com
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*