You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2021/01/04 10:35:07 UTC
[GitHub] [cloudstack] davidjumani opened a new issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
davidjumani opened a new issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566
The `memoryintfreekbs` retuned by listVirtualMachines for KVM is always greater than the total memory allocated for the VM. This is because it returns the RSS of the process running the VM (https://libvirt.org/manpages/virsh.html#dommemstat)
https://github.com/apache/cloudstack/blob/master/plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java#L3712-L3718
Need to discuss how to handle this as well as the impact on usage
##### ISSUE TYPE
<!-- Pick one below and delete the rest -->
* Bug Report
##### CLOUDSTACK VERSION
~~~
4.11+ (tested)
~~~
##### OS / ENVIRONMENT
KVM
##### STEPS TO REPRODUCE
```
(localcloud) SBCM5> > list virtualmachines filter=memory,memoryintfreekbs,memorykbs,memorytargetkbs,
{
"count": 3,
"virtualmachine": [
{
"memory": 512,
"memoryintfreekbs": 552836, <---
"memorykbs": 524288,
"memorytargetkbs": 524288
},
```
##### EXPECTED RESULTS
~~~
The used memory
~~~
##### ACTUAL RESULTS
~~~
Total memory used by the process running the VM
~~~
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
davidjumani commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754444626
So since the field's docs mention that it is the free memory inside the VM, guessing the docs should be changed or set to zero for KVM ? Right now it is greater than the total configured memory and used in the new UI while displaying the VM details
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754441686
ok, i will not make a point of this but as an enterprise cloud operator I am not at all interested in the memory usage of a vm/user, only in the amount of memory it costs me/the company. I understand public cloud wins here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754215964
> thanks. can we change the title to match the problem please. Anyone looking back through commits will be misled.
>
> I don't _think_ that the overhead to run a VM is the end-users problem, I _think_ that if I created a VM with 8GB RAM, I would expect to get charged for 8GB.
Totally agree.
by the way, there is a way to get accurate memory usage: memory ballooning.
see https://libvirt.org/formatdomain.html#memory-balloon-device
```
<devices>
<memballoon model='virtio'>
<stats period='10'/>
</memballoon>
</devices>
```
we could add a global setting to enable/disable it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] wido commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
wido commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754640904
> > thanks. can we change the title to match the problem please. Anyone looking back through commits will be misled.
> > I don't _think_ that the overhead to run a VM is the end-users problem, I _think_ that if I created a VM with 8GB RAM, I would expect to get charged for 8GB.
>
> Totally agree.
>
> by the way, there is a way to get accurate memory usage: memory ballooning.
> see https://libvirt.org/formatdomain.html#memory-balloon-device
>
> ```
> <devices>
> <memballoon model='virtio'>
> <stats period='10'/>
> </memballoon>
> </devices>
> ```
>
> we could add a global setting to enable/disable it.
Personally I'm not really a fan of all the global settings we are introducing in CloudStack. The list gets longer and longer.
That said, virtio ballooning is also something which needs to be supported by the guest. Linux does this, but Windows or BSD won't always do this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
davidjumani commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754059296
I agree as well so I gather that we have a consensus on it. The other issue is that the `memoryintfreekbs` parameter returns the wrong result (the rss instead) and it might not be possible to retrieve the free memory in the VM. What would be a good approach to go about handling it ? @PaulAngus @wido @andrijapanicsb @DaanHoogland
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
rhtyd commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-810043902
Fixed in https://github.com/apache/cloudstack/pull/4571
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
davidjumani commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-753897017
Related to https://github.com/apache/cloudstack/issues/3456
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] wido commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
wido commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754073271
Inside the VM depends on the hypervisor I guess. With KVM for example you have the Qemu Guest Agent which is needed to look inside the VM and get these results.
I don't think we can safely fetch that information from inside the VM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] PaulAngus commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
PaulAngus commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-753977449
This issue jumps between **FREE** memory and **USED** memory - which one are we talking about here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] andrijapanicsb commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754043568
agree with Paul and Wido.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] weizhouapache commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-756031074
> > > thanks. can we change the title to match the problem please. Anyone looking back through commits will be misled.
> > > I don't _think_ that the overhead to run a VM is the end-users problem, I _think_ that if I created a VM with 8GB RAM, I would expect to get charged for 8GB.
> >
> >
> > Totally agree.
> > by the way, there is a way to get accurate memory usage: memory ballooning.
> > see https://libvirt.org/formatdomain.html#memory-balloon-device
> > ```
> > <devices>
> > <memballoon model='virtio'>
> > <stats period='10'/>
> > </memballoon>
> > </devices>
> > ```
> >
> >
> > we could add a global setting to enable/disable it.
>
> Personally I'm not really a fan of all the global settings we are introducing in CloudStack. The list gets longer and longer.
>
yep. but cloudstack is a open source project.
There are some features which are used by some users but not used by other users. They might use different setting.
It is the certain result if more features and improvements are added.
> That said, virtio ballooning is also something which needs to be supported by the guest. Linux does this, but Windows or BSD won't always do this.
@wido true. In windows, user have to install virtio driver for memory ballooning. If the driver is installed incorrectly, vm might be frozen after migration.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] andrijapanicsb commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754174960
+1 on what Wido said, as this is public cloud people will be (mostly/soemtimes) running, so it's impossible to ensure specific hypervisor tools are running.
I don't see the value of monitoring the amount of RAM used inside the VM - let me user monitor his own VMs if needed (not the operator) using whatever tools they use.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] wido commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
wido commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754033022
@PaulAngus I agree. Overhead is something the operator should take into account. We at PCextreme do that. We take a safety margin on top of all the memory we allocate to VMs and which we then reserve for the Operating System, buffers and overhead.
So yes, the end-user should never be aware of the RSS on the hypervisor.
VMs using Ceph for example have a much higher overhead to due librados and librbd consuming memory under the hood.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-753959985
@PaulAngus @andrijapanicsb or any other functional eyes on this (@weizhouapache, @wido maybe from your camp), This one raises a question about billing, should an operator be able to see both memory used by the vm and by the containing process for their accounting? what is the best un-ambiguous way to define this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] wido commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
wido commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754641043
> So since the field's docs mention that it is the free memory inside the VM, guessing the docs should be changed or set to zero for KVM ? Right now it is greater than the total configured memory and used in the new UI while displaying the VM details
I would set it to 0 for KVM indeed. We can't reliably get this data.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] davidjumani commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
davidjumani commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-755885510
Added a fix to set `memoryintfreekbs` to zero if it is greater than `memorykbs`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] PaulAngus commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
PaulAngus commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-754009473
thanks. can we change the title to match the problem please. Anyone looking back through commits will be misled.
I don't _think_ that the overhead to run a VM is the end-users problem, I _think_ that if I created a VM with 8GB RAM, I would expect to get charged for 8GB.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] rhtyd closed issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
rhtyd closed issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [cloudstack] DaanHoogland commented on issue #4566: Incorrect `memoryintfreekbs` in listVirtualMachines for KVM
Posted by GitBox <gi...@apache.org>.
DaanHoogland commented on issue #4566:
URL: https://github.com/apache/cloudstack/issues/4566#issuecomment-753987187
the reported issue is about how free mem is calculated but the real issue is whether used memory as used by the VM-process as opposed to the memory used inside the vm should be attributed to the user or reported in some way at all
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org