You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Kristian Liivak <kr...@wavecom.ee.INVALID> on 2024/05/20 12:48:12 UTC
KVM FC shared storage
Hi All
Currently, we can see from the documentation that KVM supports Fiber Channel via shared mountpoints.
Can someone recommend or share their experience with usable solutions for shared mountpoint technical solutions? It seems there are quite a few options, such as shared/clustered file systems: GFS2, OCFS2, cLVM.
From the information I have found, OCFS2 seems to be the best option.
P.S. I don't want to use any distributed storage like Ceph, etc.
We are partially moving away from VMware, and we believe in FC NVMe old-school storages, which are the most reliable, in my opinion.
Lugupidamisega / Best regards,
Kristian Liivak
Tegevjuht / CEO
[ mailto:kris@wavecom.ee | kris@wavecom.ee ] | +372 5685 0001
WaveCom AS | ISO 9001, 27001 & 27017 Certified DC and Cloud services
Endla 16, Tallinn 10142 | [ http://www.wavecom.ee/ | www.wavecom.ee ] | [ http://www.facebook.com/wavecom.ee | www.facebook.com/wavecom.ee ]
Re: KVM FC shared storage
Posted by Kristian Liivak <kr...@wavecom.ee.INVALID>.
Indeed, Xenserver is good and our primary choice. We allready have it up and running..
Why i ask KVM cluster its cloudstack lacks kubernetes csi driver support for xenserver.
And we want to use kubernetes. My idea is just make one kvm cluster for kubernetes users.
Of cource we will check cloudstack csi integration, maybe we can make some changes to current csi to support xenserver..
But its wise to evaluete all options :)
Lugupidamisega / Best regards,
Kristian Liivak
Tegevjuht / CEO
[ mailto:kris@wavecom.ee | kris@wavecom.ee ] | +372 5685 0001
WaveCom AS | ISO 9001, 27001 & 27017 Certified DC and Cloud services
Endla 16, Tallinn 10142 | [ http://www.wavecom.ee/ | www.wavecom.ee ] | [ http://www.facebook.com/wavecom.ee | www.facebook.com/wavecom.ee ]
----- Original Message -----
From: "Vivek Kumar" <vi...@indiqus.com.INVALID>
To: "CloudStack Users Mailing list" <us...@cloudstack.apache.org>
Sent: Tuesday, 21 May, 2024 08:47:22
Subject: Re: KVM FC shared storage
We once tried to setup PCS cluster with GFS2 In production, and it required a lot of expertise to manage PCS cluster and our experience was very bad with that too, after a year we had to move to NFS due to so many issue. FC works better XenServer, We had almost for 5-6 year with Xenserver And FC and that too without any single downtime.
Vivek Kumar
Sr. Manager - Cloud & DevOps
TechOps | Indiqus Technologies
vivek.kumar@indiqus.com <ma...@indiqus.com>
www.indiqus.com <https://www.indiqus.com/>
> On 20 May 2024, at 9:34 PM, Andreas S. Kerber <as...@ag-trek.de.INVALID> wrote:
>
> We're running FC with OCFS2 (OS: Oracle Linux 9) on a demo 2 node cluster.
> Works kind of nice, but of course is not the same as ESXi. For now it works well enough but expanding that to a actual >10 node setup with hundreds of VMs doesn't feel right.
>
>
> Am Mon, May 20, 2024 at 03:48:12PM +0300 schrieb Kristian Liivak:
>> Hi All
>>
>> Currently, we can see from the documentation that KVM supports Fiber Channel via shared mountpoints.
>> Can someone recommend or share their experience with usable solutions for shared mountpoint technical solutions? It seems there are quite a few options, such as shared/clustered file systems: GFS2, OCFS2, cLVM.
>> From the information I have found, OCFS2 seems to be the best option.
>>
>> P.S. I don't want to use any distributed storage like Ceph, etc.
>> We are partially moving away from VMware, and we believe in FC NVMe old-school storages, which are the most reliable, in my opinion.
>>
>>
>>
>>
>> Lugupidamisega / Best regards,
>>
>> Kristian Liivak
>> Tegevjuht / CEO
>> [ mailto:kris@wavecom.ee | kris@wavecom.ee ] | +372 5685 0001
>>
>> WaveCom AS | ISO 9001, 27001 & 27017 Certified DC and Cloud services
>>
>>
>> Endla 16, Tallinn 10142 | [ http://www.wavecom.ee/ | www.wavecom.ee ] | [ http://www.facebook.com/wavecom.ee | www.facebook.com/wavecom.ee ]
>>
--
This message is intended only for the use of the individual or entity to
which it is addressed and may contain confidential and/or privileged
information. If you are not the intended recipient, please delete the
original message and any copy of it from your computer system. You are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited unless proper authorization has been
obtained for such action. If you have received this communication in error,
please notify the sender immediately. Although IndiQus attempts to sweep
e-mail and attachments for viruses, it does not guarantee that both are
virus-free and accepts no liability for any damage sustained as a result of
viruses.
Re: KVM FC shared storage
Posted by Vivek Kumar <vi...@indiqus.com.INVALID>.
We once tried to setup PCS cluster with GFS2 In production, and it required a lot of expertise to manage PCS cluster and our experience was very bad with that too, after a year we had to move to NFS due to so many issue. FC works better XenServer, We had almost for 5-6 year with Xenserver And FC and that too without any single downtime.
Vivek Kumar
Sr. Manager - Cloud & DevOps
TechOps | Indiqus Technologies
vivek.kumar@indiqus.com <ma...@indiqus.com>
www.indiqus.com <https://www.indiqus.com/>
> On 20 May 2024, at 9:34 PM, Andreas S. Kerber <as...@ag-trek.de.INVALID> wrote:
>
> We're running FC with OCFS2 (OS: Oracle Linux 9) on a demo 2 node cluster.
> Works kind of nice, but of course is not the same as ESXi. For now it works well enough but expanding that to a actual >10 node setup with hundreds of VMs doesn't feel right.
>
>
> Am Mon, May 20, 2024 at 03:48:12PM +0300 schrieb Kristian Liivak:
>> Hi All
>>
>> Currently, we can see from the documentation that KVM supports Fiber Channel via shared mountpoints.
>> Can someone recommend or share their experience with usable solutions for shared mountpoint technical solutions? It seems there are quite a few options, such as shared/clustered file systems: GFS2, OCFS2, cLVM.
>> From the information I have found, OCFS2 seems to be the best option.
>>
>> P.S. I don't want to use any distributed storage like Ceph, etc.
>> We are partially moving away from VMware, and we believe in FC NVMe old-school storages, which are the most reliable, in my opinion.
>>
>>
>>
>>
>> Lugupidamisega / Best regards,
>>
>> Kristian Liivak
>> Tegevjuht / CEO
>> [ mailto:kris@wavecom.ee | kris@wavecom.ee ] | +372 5685 0001
>>
>> WaveCom AS | ISO 9001, 27001 & 27017 Certified DC and Cloud services
>>
>>
>> Endla 16, Tallinn 10142 | [ http://www.wavecom.ee/ | www.wavecom.ee ] | [ http://www.facebook.com/wavecom.ee | www.facebook.com/wavecom.ee ]
>>
--
This message is intended only for the use of the individual or entity to
which it is addressed and may contain confidential and/or privileged
information. If you are not the intended recipient, please delete the
original message and any copy of it from your computer system. You are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited unless proper authorization has been
obtained for such action. If you have received this communication in error,
please notify the sender immediately. Although IndiQus attempts to sweep
e-mail and attachments for viruses, it does not guarantee that both are
virus-free and accepts no liability for any damage sustained as a result of
viruses.
Re: KVM FC shared storage
Posted by "Andreas S. Kerber" <as...@ag-trek.de.INVALID>.
We're running FC with OCFS2 (OS: Oracle Linux 9) on a demo 2 node cluster.
Works kind of nice, but of course is not the same as ESXi. For now it works well enough but expanding that to a actual >10 node setup with hundreds of VMs doesn't feel right.
Am Mon, May 20, 2024 at 03:48:12PM +0300 schrieb Kristian Liivak:
> Hi All
>
> Currently, we can see from the documentation that KVM supports Fiber Channel via shared mountpoints.
> Can someone recommend or share their experience with usable solutions for shared mountpoint technical solutions? It seems there are quite a few options, such as shared/clustered file systems: GFS2, OCFS2, cLVM.
> From the information I have found, OCFS2 seems to be the best option.
>
> P.S. I don't want to use any distributed storage like Ceph, etc.
> We are partially moving away from VMware, and we believe in FC NVMe old-school storages, which are the most reliable, in my opinion.
>
>
>
>
> Lugupidamisega / Best regards,
>
> Kristian Liivak
> Tegevjuht / CEO
> [ mailto:kris@wavecom.ee | kris@wavecom.ee ] | +372 5685 0001
>
> WaveCom AS | ISO 9001, 27001 & 27017 Certified DC and Cloud services
>
>
> Endla 16, Tallinn 10142 | [ http://www.wavecom.ee/ | www.wavecom.ee ] | [ http://www.facebook.com/wavecom.ee | www.facebook.com/wavecom.ee ]
>
Re: KVM FC shared storage
Posted by "Dietrich, Alex" <ad...@ussignal.com.INVALID>.
Hello Kristian,
Take this perspective with a grain of salt given our time spent on this was in a proof-of-concept deployment.
We tested iSCSI connectivity with OCFS2 as the underlying technology to provide the shared mount point. We found that throughout the course of host reboots, the reliability of varying components re-establishing connectivity was not up to par with the reliability of the underlying storage technology (iSCSI). In a few cases, I found OCFS2 unable to establish peers with the other KVM hypervisors, which required rebuilding the OCFS2 cluster to re-establish connectivity.
The other mechanisms to create a shared mountpont were not tested as we pivoted away from the idea given our experience with OCFS2.
Thanks,
Alex
[__tpx__]
From: Kristian Liivak <kr...@wavecom.ee.INVALID>
Date: Monday, May 20, 2024 at 8:48 AM
To: users <us...@cloudstack.apache.org>
Subject: KVM FC shared storage
EXTERNAL
Hi All
Currently, we can see from the documentation that KVM supports Fiber Channel via shared mountpoints.
Can someone recommend or share their experience with usable solutions for shared mountpoint technical solutions? It seems there are quite a few options, such as shared/clustered file systems: GFS2, OCFS2, cLVM.
From the information I have found, OCFS2 seems to be the best option.
P.S. I don't want to use any distributed storage like Ceph, etc.
We are partially moving away from VMware, and we believe in FC NVMe old-school storages, which are the most reliable, in my opinion.
Lugupidamisega / Best regards,
Kristian Liivak
Tegevjuht / CEO
kris@wavecom.ee<ma...@wavecom.ee> | +372 5685 0001
[cid:58154b1f50b08ffef0dccd912a90853b36251804@zimbra]
WaveCom AS | ISO 9001, 27001 & 27017 Certified DC and Cloud services
Endla 16, Tallinn 10142 | www.wavecom.ee<https://urldefense.com/v3/__http:/www.wavecom.ee/__;!!P9cq_d3Gyw!kdPr3d5okgMIjSmm_GQ6jXtU_qJ54d1A9xUNCpf5TGe0rXZUF5J9yUPsKkmjz8b9Q0GLE4FMGhTexsqbkekSXA$> | www.facebook.com/wavecom.ee<https://urldefense.com/v3/__http:/www.facebook.com/wavecom.ee__;!!P9cq_d3Gyw!kdPr3d5okgMIjSmm_GQ6jXtU_qJ54d1A9xUNCpf5TGe0rXZUF5J9yUPsKkmjz8b9Q0GLE4FMGhTexspKUxaBBA$>
RE: KVM FC shared storage
Posted by Alex Mattioli <Al...@shapeblue.com>.
If you are 100% FC/NVME then OCFS2 is probably the best (or least bad) option.
I personally always tried to stick to NFS for KVM, it just works.
Reliability wise, I personally consider CEPH to be more reliable (and supportable) than OCFS2.
Regards
Alex
From: Kristian Liivak <kr...@wavecom.ee.INVALID>
Sent: Monday, May 20, 2024 2:48 PM
To: users <us...@cloudstack.apache.org>
Subject: KVM FC shared storage
Hi All
Currently, we can see from the documentation that KVM supports Fiber Channel via shared mountpoints.
Can someone recommend or share their experience with usable solutions for shared mountpoint technical solutions? It seems there are quite a few options, such as shared/clustered file systems: GFS2, OCFS2, cLVM.
From the information I have found, OCFS2 seems to be the best option.
P.S. I don't want to use any distributed storage like Ceph, etc.
We are partially moving away from VMware, and we believe in FC NVMe old-school storages, which are the most reliable, in my opinion.
Lugupidamisega / Best regards,
Kristian Liivak
Tegevjuht / CEO
kris@wavecom.ee<ma...@wavecom.ee> | +372 5685 0001
[cid:image001.png@01DAAACA.9761B690]
WaveCom AS | ISO 9001, 27001 & 27017 Certified DC and Cloud services
Endla 16, Tallinn 10142 | www.wavecom.ee<http://www.wavecom.ee/> | www.facebook.com/wavecom.ee<http://www.facebook.com/wavecom.ee>