Re: VCL, ZFS/clones, VMs and provisioning to ESX

[Aaron Peeler <aa...@ncsu.edu>]

Hi,
Sorry for the delayed response.

I think this is pretty cool approach and would like to see more about 
it. I realize that there is not a one-size fits all in the Provisioning 
Module when it comes to storage options like this.

Ideally what needs to happen is we need modularize the storage 
components. The would keep general vmware based common tasks with in the 
provisioning modules(i.e. register,unregister, start, stop, suspend, 
etc) an allow for different storage options that could match one's 
infrastructure. But in the meantime until storage is modularized, your 
approach makes sense.

Aaron

On 1/27/10 8:14 AM, Matjaž Pančur wrote:
> We currently use ZFS (currently Nexenta's Developer Edition, www.nexenta.com) as a NFS datastore. All works well, however copying VMDKs during ESX provisioning is bothering me a bit (we use cheap SATA based HW, so scalability of large number of simultaneous requests is an issue). However, with ZFS's unlimited snapshots/clones (clone is a RW snapshot), copying is almost instantaneous.
>
> I'm creating an esxzfs.pm which is using ZFS snapshots and clones to "thin provision" (clone) VM's for ESX servers, basically the same as esxthin.pm for NetApp HW. Since ZFS has limitations on cloning (can't do file level cloning, only filesystem snapshot/cloning), things are not so straightforward as esxthin.pm implementation.
>
> ESX NFS client (and Nexenta's NFS server) doesn't support automatic mounting of subfilesystems, so you have to mount each VCL's slot (computer) individually.
>
>
> vcl/golden/image1
>                    /image2
>      /inuse/slot1
>                /slot2
>
> In ordinary case (one filesystem - vcl, shared as NFS), you mount [VCL] in ESX and you see golden, inuse and all subfolders, use ssh cp and rm for image manipulation, ... and all is ok.
>
> ZFS supports only filesystem snapshoting/cloning, however snapshoting/cloning EVERYTHING for a single VM clone is not practical. So the basic idea is:
>
> * every folder is a distinct filesystem (in ZFS/Nexenta, folder=separate FS)
> * snapshot golden/imagex (e.g. golden/imagex@now) and clone it to inuse/sloty
> * mount /inuse/sloty on a ESX as [sloty]
> * register and start VM from [sloty]
>
> Limitations:
> * ESX supports only 64 NFS mounts, so maximum number of slots (computers) on a single ESX server is 64. (Which is ok in my situation).
> * can't mix "normal" esx and esxzfs provisioning in a single NFS server (if you mount only "root" vcl, you do't see files in golden/imagex or inuse/slotx, you have to mount slotx or imagey explicitly). I'm ok with that too.
>
>
> Thoughts, suggestions?


-- 

Aaron Peeler
Program Manager
Virtual Computing Lab
NC State University
aaron_peeler@ncsu.edu
919-513-4571