You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Granwille Strauss <gr...@namhost.com.INVALID> on 2024/04/03 09:06:52 UTC
Re: Cloudstack Still Trying to connect to secondary storage NFS after removing it.
Hi
I want to pick this up again, this issue is happening again. I went as
far as destroying SSVM but the management server always tries to connect
back to this NFS that no longer exists, which hampers other services
from running in cloudstack.
Is there a way I can clean out NFS mounts properly? Because this NFS
mount does not exist in my /ect/fstab on my management server, its
cloudstack itself that keeps creating it for some reason.
On 11/22/23 11:43, Jithin Raju wrote:
> Granwille,
>
> The ISO that is mentioned in the below log snippets shows SSVM (not management server itself) is trying to download the file to NFS server 192.168.50.2. Is it the secondary storage you removed? If yes restart the SSVM. If management server is still attempting to connect to this NFS server/share you could unmount that share on the management server.
>
>
> -Jithin
>
> From: Granwille Strauss<gr...@namhost.com.INVALID>
> Date: Tuesday, 21 November 2023 at 9:39 AM
> To:users@cloudstack.apache.org <us...@cloudstack.apache.org>
> Subject: Cloudstack Still Trying to connect to secondary storage NFS after removing it.
>
> Hi Guys
>
> My cloud stack management server still wants to connect to a secondary NFS server. We have shutdown the NFS storage and we removed it from Cloudstack management via UI in the secondary storage area, we made sure to migrate all its existing data to another secondary NFS prior.
>
> But now, when I load new templates or ISO the download fails and says it times out:
> 2023-11-21 10:00:02,665 DEBUG [c.c.a.t.Request] (qtp1312381159-9364:ctx-78f2d817 ctx-3bf3aa88) (logid:c4456add) Seq 57-4037477065937751626: Sending { Cmd , MgmtId: 66988330791812, via: 57(s-189-VM), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DownloadCommand":{"hvm":"true","description":"Ubuntu 22.04 LTS - Server Image","maxDownloadSizeInBytes":"(50,00 GB) 53687091200","id":"237","resourceType":"TEMPLATE","installPath":"template/tmpl/2/237","_store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://192.168.50.2/mnt/datasecond","_role":"Image"}},"url":"https://releases.ubuntu.com/jammy/ubuntu-22.04.3-live-server-amd64.iso"<https://releases.ubuntu.com/jammy/ubuntu-22.04.3-live-server-amd64.iso>,"format":"ISO","accountId":"2","name":"237-2-7d385757-227c-3660-a56c-1aaae59cacc3","secUrl":"nfs://192.168.50.2/mnt/datasecond","wait":"0","bypassHostMaintenance":"false"}}] }
>
>
> 2023-11-21 10:00:17,855 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-3:null) (logid:) SeqA 49-685551: Sending Seq 49-685551: { Ans: , MgmtId: 66988330791812, via: 49, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
> 2023-11-21 10:00:18,877 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-b3f89e5f) (logid:2e6365a0) Enabled non-edge zones available for scan: 16
> 2023-11-21 10:00:18,880 DEBUG [c.c.s.StatsCollector] (secstorage-1:ctx-b3f89e5f) (logid:2e6365a0) Verifying image storage [28]. Capacity: total=[223 GB], used=[33 MB], threshold=[89.99999761581421%].
> 2023-11-21 10:00:18,881 DEBUG [c.c.s.StatsCollector] (secstorage-1:ctx-b3f89e5f) (logid:2e6365a0) Verifying image storage [29]. Capacity: total=[870 GB], used=[49 GB], threshold=[89.99999761581421%].
> 2023-11-21 10:00:18,884 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-b3f89e5f) (logid:2e6365a0) Zone [16] is ready to launch secondary storage VM.
>
> 2023-11-21 10:00:32,659 DEBUG [c.c.s.d.DownloadListener] (Timer-7:ctx-b1d926d0) (logid:2a0029c2) Scheduling timeout at 30000 ms, TEMPLATE: 237 at host 57
> 2023-11-21 10:00:32,659 DEBUG [c.c.s.d.DownloadListener] (Timer-7:ctx-b1d926d0) (logid:2a0029c2) NOT_DOWNLOADED first timeout: checking again , TEMPLATE: 237 at host 57
>
> Cloudstack server via dmesg also shows it remains trying to connect to the server:
> [15529788.049090] sh (779409): drop_caches: 3
> [15529800.675531] nfs: server <IP Address> not responding, timed out
> [15529805.476018] nfs: server <IP Address> not responding, timed out
> [15529814.180397] nfs: server <IP Address> not responding, timed out
> [15529847.965197] sh (785469): drop_caches: 3
>
>
> Was there a correct way I had to process this? Or am I missing something?
> --
>
> Regards / Groete
> [Image removed by sender.]<https://www.namhost.com/>
> Granwille Strauss // Senior Systems Admin
>
> e:granwille@namhost.com<ma...@namhost.com>
> m: +264 81 323 1260<tel:+264813231260>
> w:www.namhost.com<https://www.namhost.com/>
>
> [Image removed by sender.]<https://www.facebook.com/namhost>[Image removed by sender.]<https://twitter.com/namhost>[Image removed by sender.]<https://www.instagram.com/namhostinternetservices/>[Image removed by sender.]<https://www.linkedin.com/company/namhos>[Image removed by sender.]<https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
>
> [Image removed by sender.]<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
> Namhost Internet Services (Pty) Ltd,
>
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
> The content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy
>
> [Image removed by sender. Powered by AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>
>
>
>
--
Regards / Groete
<https://www.namhost.com> Granwille Strauss // Senior Systems Admin
*e:* granwille@namhost.com
*m:* +264 81 323 1260 <tel:+264813231260>
*w:* www.namhost.com <https://www.namhost.com/>
<https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
Namhost Internet Services (Pty) Ltd,
24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it by
mistake, please inform us by email reply and then delete the message. It
is forbidden to copy, forward, or in any way reveal the contents of this
message to anyone without our explicit consent. The integrity and
security of this email cannot be guaranteed over the Internet.
Therefore, the sender will not be held liable for any damage caused by
the message. For our full privacy policy and disclaimers, please go to
https://www.namhost.com/privacy-policy
Powered by AdSigner
<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>