You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2020/03/05 17:48:51 UTC
[GitHub] [cloudstack] andrijapanicsb opened a new issue #3942: Host
allocated RAM reported when no running VMs
andrijapanicsb opened a new issue #3942: Host allocated RAM reported when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942
**Problem**: host reported as being fully RAM allocated when all VMs are stopped on this host.
Reproduce:
Host has 6 VMs running.
Stop all 6 VMs - zero VMs running on this host.
Fetch the latest metrics via Dashboard "Fetch latest" button.
Go to host, click on it, open the Statistics tab - it will show RAM memory as allocated (sum of RAM memory of all VMs) - so no new VMs cant be started on it.
I've traced this to the last_host_id value in vm_instance table.
Test workaround:
- Open DB editor, find all VMs where last_host_id = <hostID> and change this to another host ID - so that no VMs are referencing this particular host. Fetch the latest statistics again - it will show reduction of RAM allocated.
CloudStack Master (4.14) (one of the PRs - needs to be tested in the actual Master as well)
KVM (Ubunut18, should not be relevant)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] andrijapanicsb commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-596426448
Thanks @luhaijiao - makes sense for some users, but not in general (in my head at least) if the VM is stopped, it's stopped - all the VMs should compete for CPU/RAM resources... but I see this is from before 3.x release... thx again both.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] weizhouapache commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-596455530
> @weizhouapache does it sound OK, if we change this setting to 0 as the default value - do you see any consequences?
@andrijapanicsb even if it is set to 0, the resource is not released immediately.
it will be released by thread HostReservationReleaseChecker which runs every 5 minutes (configured by global setting host.reservation.release.period).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] andrijapanicsb commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-595372260
Confirmed the issue in 4.13
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] andrijapanicsb edited a comment on issue #3942: Host
reports RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
andrijapanicsb edited a comment on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-598692706
My attempt to avoid edge case is useless here, as the implementation of those capacity cheks is such that I can't accomplish a "fix" - so closing this Issues and the corresponding PR #3961
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] andrijapanicsb commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-598692706
My attempt to avoid edge case is useless here, as the implementation of those capacity cheks is such that I can't acomplish a "fix" - so closing this Issues and the coresponding PR #3961
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] weizhouapache commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-595377101
@andrijapanicsb might the resource be reserved for stopped vm?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] andrijapanicsb commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-595367577
I've observed this in the Rolling Maintenance PR - so need to test Master as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] weizhouapache commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
weizhouapache commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-595414813
@andrijapanicsb
just tested it, when vm is stopped, the resources (cpu/ram) will be reserved for 1 hour (configurable in global setting).
1. before stopping vm
```
mysql> select host_id,used_capacity,reserved_capacity,total_capacity,capacity_type from op_host_capacity where host_id=14;
+---------+---------------+-------------------+----------------+---------------+
| host_id | used_capacity | reserved_capacity | total_capacity | capacity_type |
+---------+---------------+-------------------+----------------+---------------+
| 14 | 3221225472 | 0 | 7297568768 | 0 |
| 14 | 2500 | 0 | 9196 | 1 |
| 14 | 4 | 0 | 4 | 90 |
+---------+---------------+-------------------+----------------+---------------+
3 rows in set (0.01 sec)
```
2. after stopping vm
```
mysql> select host_id,used_capacity,reserved_capacity,total_capacity,capacity_type from op_host_capacity where host_id=14;
+---------+---------------+-------------------+----------------+---------------+
| host_id | used_capacity | reserved_capacity | total_capacity | capacity_type |
+---------+---------------+-------------------+----------------+---------------+
| 14 | 2684354560 | 536870912 | 7297568768 | 0 |
| 14 | 2000 | 500 | 9196 | 1 |
| 14 | 3 | 1 | 4 | 90 |
+---------+---------------+-------------------+----------------+---------------+
3 rows in set (0.01 sec)
```
3. the reservation will be released after 1 hour.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] andrijapanicsb closed issue #3942: Host reports RAM
allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
andrijapanicsb closed issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] andrijapanicsb commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-598122196
Ah - that sounds so wrong... thx
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] luhaijiao commented on issue #3942: Host reports RAM
allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
luhaijiao commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-596044021
I guess the reservation is to handle the situation that the stopped VM need start again but its capacity has been taken over by new VMs which may cause the start failure and bad user experience.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] andrijapanicsb commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-595371430
@weizhouapache haven't you fixed something similar - so that the stopped VMs doesn't "count" against the capacities?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] andrijapanicsb commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-595661126
Thanks @weizhouapache - any idea what's the reason to reserve those in the first place (what's the usefulness of reserving capacities?)
I'm testing the Rolling maintenance with KVM and this affects/cause issues while testing when the cluster is tight on resources (or only a very few available hosts, etc).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [cloudstack] andrijapanicsb commented on issue #3942: Host reports
RAM allocated, when no running VMs
Posted by GitBox <gi...@apache.org>.
andrijapanicsb commented on issue #3942: Host reports RAM allocated, when no running VMs
URL: https://github.com/apache/cloudstack/issues/3942#issuecomment-596428160
@weizhouapache does it sound OK, if we change this setting to 0 as the default value - do you see any consequences?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services