You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by "akrasnov-drv (via GitHub)" <gi...@apache.org> on 2025/02/26 13:16:31 UTC
[D] VM Disk performance issue [cloudstack]
GitHub user akrasnov-drv created a discussion: VM Disk performance issue
Hi,
We see degraded performance in our VMs in CloudStack (4.20.0.0) during compilation and other heavy I/O operations.
I set our compute offerings to use local disk, nevertheless when there are 20-40 VMs per host, it does not work well.
As I understand it happens because of shared base disk architecture used by CloudStack with QCOW images.
According to information I found for QCOW:
Case-Based Limits:
• Light workloads: 10-20 images sharing one base can work fine.
• Heavy workloads (e.g., database, high IOPS): Keep it below 5 per base.
• Cloud setups (OpenStack, CloudStack): Optimized storage backends (Ceph, LVM) are preferable.
Please advise if
- there is a way to limit number of images per shared base (or completely change this to independent disks)
- there is a way to define compute offering initially with 2 disks - 1st root with shared base from template, 2nd independent and empty.
Thanks,
Alex.
GitHub link: https://github.com/apache/cloudstack/discussions/10468
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] VM Disk performance issue [cloudstack]
Posted by "akrasnov-drv (via GitHub)" <gi...@apache.org>.
GitHub user akrasnov-drv added a comment to the discussion: VM Disk performance issue
Oh great! Thanks @rajujith
I'll try and test.
I've got hope :)
GitHub link: https://github.com/apache/cloudstack/discussions/10468#discussioncomment-12365690
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] VM Disk performance issue [cloudstack]
Posted by "akrasnov-drv (via GitHub)" <gi...@apache.org>.
GitHub user akrasnov-drv edited a comment on the discussion: VM Disk performance issue
Hello @NuxRo,
Thank you for your reply.
I hoped to get some other answer, e.g. that I missed some way to do the desired changes.
Till now we used own VMs with dedicated disks and pinned cores (core pass-through with no reuse, grouped per numa). These VMs accomplish our tasks significantly faster than CloudStack VMs with the same resources on the same servers. For now I don't think the difference could rise just because of CPU cores (I also configured cpu pass-through, though without numa optimization), so only the disks are left.
I'll try to experiment with this more. For now I believe the problem is in layered disks.
If anybody gets ideas on optimize disks in CloudStack, please share.
Thanks,
Alex.
GitHub link: https://github.com/apache/cloudstack/discussions/10468#discussioncomment-12328928
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] VM Disk performance issue [cloudstack]
Posted by "rajujith (via GitHub)" <gi...@apache.org>.
GitHub user rajujith added a comment to the discussion: VM Disk performance issue
@akrasnov-drv Could you create a new compute offering with the storage provisioning type 'Fat provisioning'? this should create dedicated disks for your instances created by CloudStack. Will that make the instance configuration close to the external VMs so that you can compare the performance?
GitHub link: https://github.com/apache/cloudstack/discussions/10468#discussioncomment-12334080
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] VM Disk performance issue [cloudstack]
Posted by "NuxRo (via GitHub)" <gi...@apache.org>.
GitHub user NuxRo added a comment to the discussion: VM Disk performance issue
Hello,
I have never noticed issues using linked-clones and I suspect the culprit may be something else, perhaps environmental, otherwise we would have had people complain. Linked-clones has always been the default.
To answer your question, you cannot limit the number of images per shared image (ie how many VMs you can spawn from a given template).
Also, you cannot define a compute offering which contains 2 disks, you'd have to create the 2nd disk as part of the creation procedure.
It'd be worth doing some in-depth monitoring of the hypervisor, especially I/O metrics and see what is going on when you feel like it's slow etc.
GitHub link: https://github.com/apache/cloudstack/discussions/10468#discussioncomment-12327876
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] VM Disk performance issue [cloudstack]
Posted by "akrasnov-drv (via GitHub)" <gi...@apache.org>.
GitHub user akrasnov-drv edited a comment on the discussion: VM Disk performance issue
Hello @NuxRo,
Thank you for your reply.
I hoped to get some other answer, e.g. that I missed some way to do the desired changes.
Till now we used own VMs with dedicated disks and pinned cores (core pass-through with no reuse, grouped per numa). These VMs accomplish our tasks significantly faster than CloudStack VMs with the same resources on the same servers. For now I don't think the difference could rise just because of CPU cores (I also configured cpu pass-through, though without numa optimization), so only the disks are left.
I'll try to experiment with this more. For now I believe the problem is in layered disks.
If anybody gets ideas on optimizing disks in CloudStack, please share.
Thanks,
Alex.
GitHub link: https://github.com/apache/cloudstack/discussions/10468#discussioncomment-12328928
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] VM Disk performance issue [cloudstack]
Posted by "akrasnov-drv (via GitHub)" <gi...@apache.org>.
GitHub user akrasnov-drv closed a discussion: VM Disk performance issue
Hi,
We see degraded performance in our VMs in CloudStack (4.20.0.0) during compilation and other heavy I/O operations.
I set our compute offerings to use local disk, nevertheless when there are 20-40 VMs per host, it does not work well.
As I understand it happens because of shared base disk architecture used by CloudStack with QCOW images.
According to information I found for QCOW:
Case-Based Limits:
• Light workloads: 10-20 images sharing one base can work fine.
• Heavy workloads (e.g., database, high IOPS): Keep it below 5 per base.
• Cloud setups (OpenStack, CloudStack): Optimized storage backends (Ceph, LVM) are preferable.
Please advise if
- there is a way to limit number of images per shared base (or completely change this to independent disks)
- there is a way to define compute offering initially with 2 disks - 1st root with shared base from template, 2nd independent and empty.
Thanks,
Alex.
GitHub link: https://github.com/apache/cloudstack/discussions/10468
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] VM Disk performance issue [cloudstack]
Posted by "akrasnov-drv (via GitHub)" <gi...@apache.org>.
GitHub user akrasnov-drv edited a comment on the discussion: VM Disk performance issue
Hello @NuxRo,
Thank you for your reply.
I hoped to get some other answer, e.g. that I missed some way to do the desired changes.
Till now we used own VMs with dedicated disks and pinned cores (core pass-through with no reuse grouped per numa). These VMs accomplish our tasks significantly faster than CloudStack VMs with the same resources on the same servers. For now I don't think the difference could rise just because of CPU cores (I also configured cpu pass-through, though without numa optimization), so only the disks are left.
I'll try to experiment with this more. For now I believe the problem is in layered disks.
If anybody gets ideas on optimize disks in CloudStack, please share.
Thanks,
Alex.
GitHub link: https://github.com/apache/cloudstack/discussions/10468#discussioncomment-12328928
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] VM Disk performance issue [cloudstack]
Posted by "akrasnov-drv (via GitHub)" <gi...@apache.org>.
GitHub user akrasnov-drv added a comment to the discussion: VM Disk performance issue
Hello @NuxRo,
Thank you for your reply.
I hoped to get some other answer, e.g. that I missed some way to do the desired changes.
Till now we used own VMs with dedicated disks and pinned cores (core pass-through with no reuse grouped per numa). These VMs accomplish our tasks significantly faster than CloudStack VMs with the same resources. For now I don't think the difference could rise just because of CPU cores (I also configured cpu pass-through, though without numa optimization), so only the disks are left.
I'll try to experiment with this more. For now I believe the problem is in layered disks.
If anybody gets ideas on optimize disks in CloudStack, please share.
Thanks,
Alex.
GitHub link: https://github.com/apache/cloudstack/discussions/10468#discussioncomment-12328928
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] VM Disk performance issue [cloudstack]
Posted by "akrasnov-drv (via GitHub)" <gi...@apache.org>.
GitHub user akrasnov-drv added a comment to the discussion: VM Disk performance issue
Thank you.
Fat disk really does what I wanted and solves the issue (don't know how I overlooked it)
GitHub link: https://github.com/apache/cloudstack/discussions/10468#discussioncomment-12390061
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org