You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Yordan Kostov <Yo...@NSOGROUP.COM> on 2021/10/04 15:25:41 UTC
VR swap on disk
Hey everyone,
Environment is 4.15 ACS + XCP-NG 8.2 on fiber shared storage.
I notice a strange VR behavior happening once in a while (~10 mins or so) . The virtual router is generating significant amount of usage on the storage for about 1 min or so.
Storage usage window corelates with CPU 100% usage as well.
After some troubleshooting - the process that does the CPU load is kswapd0. it looks like the VR is swapping memory on the storage.
An error message pops up once in a while in the cli - https://imgur.com/a/K2Tthi0
The disk of the VR is not full - https://imgur.com/37JU1Dg
It has happened a few times now and usually what I do is delete the VR and let ACS put a new instance, but that postpones the issue and does not resolve it.
Any idea of the root cause and how to fix ?
Regards,
Jordan
RE: VR swap on disk
Posted by Yordan Kostov <Yo...@NSOGROUP.COM>.
Thanks Wei,
After some troubleshooting. Journal gives the following errors:
- user@0.service: Failed at step PAM spawning
- Failed to start User Manager for UID 0
This correlates with some systemd articles: https://github.com/coreos/bugs/issues/1498
Anyway I will do as you say and may be update the VRs to latest version as it may be indeed systemd issue.
Regards,
Jordan
-----Original Message-----
From: Wei ZHOU <us...@gmail.com>
Sent: Monday, October 4, 2021 8:24 PM
To: users <us...@cloudstack.apache.org>
Subject: Re: VR swap on disk
[X] This message came from outside your organization
Hi Jordan,
I have no idea what caused the issue. The default cpu/ram for virtual routers is small, but enough for most cases.
There are two options to change the service offering of VR:
(1) create a new network offering, with specified router offering, then update the network to the new offering. It looks a bit complicated. you can also choose option 2:
(2) create a new system service offering, go to account settings and change router.service.offering to the UUID of the new service offering. restart network with cleanup.
Hope it helps.
-Wei
On Mon, 4 Oct 2021 at 17:25, Yordan Kostov <Yo...@nsogroup.com> wrote:
> Hey everyone,
>
> Environment is 4.15 ACS + XCP-NG 8.2 on fiber shared
> storage.
>
> I notice a strange VR behavior happening once in a
> while
> (~10 mins or so) . The virtual router is generating significant amount
> of usage on the storage for about 1 min or so.
> Storage usage window corelates with CPU 100% usage as well.
>
> After some troubleshooting - the process that does the
> CPU load is kswapd0. it looks like the VR is swapping memory on the storage.
> An error message pops up once in a while in the cli -
> https://urldefense.com/v3/__https://imgur.com/a/K2Tthi0__;!!A6UyJA!wE9xCvCO5LnRzMg-atmrahnfDUPxs2Dcq7L1blk6uqaal0o-Hm1bkFtCpg8phWDsQ9orQ2-jpMv2$
> The disk of the VR is not full -
> https://urldefense.com/v3/__https://imgur.com/37JU1Dg__;!!A6UyJA!wE9xC
> vCO5LnRzMg-atmrahnfDUPxs2Dcq7L1blk6uqaal0o-Hm1bkFtCpg8phWDsQ9orQ7jyX2H
> V$
>
> It has happened a few times now and usually what I do
> is delete the VR and let ACS put a new instance, but that postpones
> the issue and does not resolve it.
> Any idea of the root cause and how to fix ?
>
> Regards,
> Jordan
>
>
Re: VR swap on disk
Posted by Wei ZHOU <us...@gmail.com>.
Hi Jordan,
I have no idea what caused the issue. The default cpu/ram for virtual
routers is small, but enough for most cases.
There are two options to change the service offering of VR:
(1) create a new network offering, with specified router offering, then
update the network to the new offering. It looks a bit complicated. you can
also choose option 2:
(2) create a new system service offering, go to account settings and change
router.service.offering to the UUID of the new service offering. restart
network with cleanup.
Hope it helps.
-Wei
On Mon, 4 Oct 2021 at 17:25, Yordan Kostov <Yo...@nsogroup.com> wrote:
> Hey everyone,
>
> Environment is 4.15 ACS + XCP-NG 8.2 on fiber shared
> storage.
>
> I notice a strange VR behavior happening once in a while
> (~10 mins or so) . The virtual router is generating significant amount of
> usage on the storage for about 1 min or so.
> Storage usage window corelates with CPU 100% usage as well.
>
> After some troubleshooting - the process that does the CPU
> load is kswapd0. it looks like the VR is swapping memory on the storage.
> An error message pops up once in a while in the cli -
> https://imgur.com/a/K2Tthi0
> The disk of the VR is not full - https://imgur.com/37JU1Dg
>
> It has happened a few times now and usually what I do is
> delete the VR and let ACS put a new instance, but that postpones the issue
> and does not resolve it.
> Any idea of the root cause and how to fix ?
>
> Regards,
> Jordan
>
>