KVM storage solution

[Emir Imamagic <ei...@srce.hr>]

Hello,

we are currently running VCL on top of VMware ESXi 4.1 and are planning 
to migrate to KVM. Main reason is that 4.1 does not support some of 
newer OS-es (e.g. Win 8) and ESXi 5 memory limitations are too 
restrictive for our servers.

Currently we have 4 VMware ESXi servers which are using iSCSI storage 
for both datastore and vmstore. For datastore we benefit from the fact 
that VMware VMFS is inherently clustered, i.e. i is able to mount same 
LUN from multiple servers.

In case of KVM we obviously need to use something else, either NFS or 
one of the server-less systems (e.g. GFS or OCFS). Our storage system 
does not provide NFS interface so we would need to set up NFS servers. 
This solution introduces another component (and cost) in system and we 
would like to avoid it. With GFS we had some performance issues in the 
past, and OCFS we haven't tested yet. We also considered using something 
more exotic like MooseFS, Gluster or CEPH, however for all of them we 
need to buy more servers, which is currently not an option.

I would like to kindly ask other KVM-based VCL instances to share their 
experiences, what do you use for datastore?

Thanks in advance
-- 
Emir Imamagic
SRCE - University of Zagreb University Computing Centre, www.srce.unizg.hr
Emir.Imamagic@srce.hr, tel: +385 1 616 5809, fax: +385 1 616 5559

Re: KVM storage solution

[Dmitri Chebotarov <dc...@gmu.edu>]

Emir

Mapping iSCSI volumes to one of the physical servers (KVM host) and then exporting filesystem via NFS will be a good option in your case. This essentially makes one of the hosts to be the NFS server, but you can still use it as KVM host in VCL. 
If you have enough local storage on each server you can configure VCL to store running images on local storage - this will improve performance, since NFS will be used mostly as read-only, except when you update an image. And if you trust your network use 'udp' mount option on NFS client, which will make it even faster.

Thanks. 

On Jul 11, 2013, at 14:16 , Emir Imamagic <ei...@srce.hr> wrote:

> Hello,
> 
> we are currently running VCL on top of VMware ESXi 4.1 and are planning to migrate to KVM. Main reason is that 4.1 does not support some of newer OS-es (e.g. Win 8) and ESXi 5 memory limitations are too restrictive for our servers.
> 
> Currently we have 4 VMware ESXi servers which are using iSCSI storage for both datastore and vmstore. For datastore we benefit from the fact that VMware VMFS is inherently clustered, i.e. i is able to mount same LUN from multiple servers.
> 
> In case of KVM we obviously need to use something else, either NFS or one of the server-less systems (e.g. GFS or OCFS). Our storage system does not provide NFS interface so we would need to set up NFS servers. This solution introduces another component (and cost) in system and we would like to avoid it. With GFS we had some performance issues in the past, and OCFS we haven't tested yet. We also considered using something more exotic like MooseFS, Gluster or CEPH, however for all of them we need to buy more servers, which is currently not an option.
> 
> I would like to kindly ask other KVM-based VCL instances to share their experiences, what do you use for datastore?
> 
> Thanks in advance
> -- 
> Emir Imamagic
> SRCE - University of Zagreb University Computing Centre, www.srce.unizg.hr
> Emir.Imamagic@srce.hr, tel: +385 1 616 5809, fax: +385 1 616 5559
> 



--
Thank you,

Dmitri Chebotarov
VCL Sys Eng, Engineering & Architectural Support, TSD - Ent Servers & Messaging
223 Aquia Building, Ffx, MSN: 1B5
Phone: (703) 993-6175 | Fax: (703) 993-3404

Re: KVM storage solution

[Peter Dimitrios <pe...@gmail.com>]

 With only 4 physical vmhost servers, you should be able to tune
achieve good performance with NFS, even if you run an NFS server in a
regular linux node. For best performance, the NFS server should be on
a bare-metal OS (not inside a VM), so that it can shovel bytes from
disk to network as quickly as possible.    You can experiment with
some of the fancier filesystems,  but performance essentially depends
on how well the NFS server can handle the physical spindles you have.
   You can handle physical storage with regular LVM tools, and watch
your network bandwidth between the nodes.   Use copy-on-write VM
clones (qcow2), so the NFS server can cache disk blocks from the
master image to serve them out to many VMs.

Re: KVM storage solution

["Pablo J. Rebollo-Sosa" <pa...@upr.edu>]

We are on the same process now.  Our testing environment is using GFS2 on a V7000 iSCSI LUN.  I will share more information when return from vacations.

Pablo

Sent from my iPhone

On Jul 11, 2013, at 2:16 PM, Emir Imamagic <ei...@srce.hr> wrote:

> Hello,
> 
> we are currently running VCL on top of VMware ESXi 4.1 and are planning to migrate to KVM. Main reason is that 4.1 does not support some of newer OS-es (e.g. Win 8) and ESXi 5 memory limitations are too restrictive for our servers.
> 
> Currently we have 4 VMware ESXi servers which are using iSCSI storage for both datastore and vmstore. For datastore we benefit from the fact that VMware VMFS is inherently clustered, i.e. i is able to mount same LUN from multiple servers.
> 
> In case of KVM we obviously need to use something else, either NFS or one of the server-less systems (e.g. GFS or OCFS). Our storage system does not provide NFS interface so we would need to set up NFS servers. This solution introduces another component (and cost) in system and we would like to avoid it. With GFS we had some performance issues in the past, and OCFS we haven't tested yet. We also considered using something more exotic like MooseFS, Gluster or CEPH, however for all of them we need to buy more servers, which is currently not an option.
> 
> I would like to kindly ask other KVM-based VCL instances to share their experiences, what do you use for datastore?
> 
> Thanks in advance
> -- 
> Emir Imamagic
> SRCE - University of Zagreb University Computing Centre, www.srce.unizg.hr
> Emir.Imamagic@srce.hr, tel: +385 1 616 5809, fax: +385 1 616 5559