You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by "UAnton (via GitHub)" <gi...@apache.org> on 2025/03/12 11:37:27 UTC

[D] Secondry Storage NFS v3 [cloudstack]

GitHub user UAnton created a discussion: Secondry Storage NFS v3

### problem

KVM nodes can't deploy Service VMs.

```
2025-03-11 21:32:30,235 ERROR [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-28:[ctx-5b29434d, job-113/job-141, ctx-9e239170]) (logid:cbf99996) Unable to create volume [{"name":"ROOT-38","uuid":"32eba291-1115-4dfe-b90c-a64de46a2de4"}] due to [com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: internal error: Child process (/usr/bin/mount -o nodev,nosuid,noexec 10.100.7.4:/secondary/template/tmpl/1/3 /mnt/3571e6e5-e631-3d46-8a7e-b2b66f8f4bc4) unexpected exit status 32: mount.nfs: Protocol not supported
].
2025-03-11 21:32:30,235 WARN  [c.c.v.ClusteredVirtualMachineManagerImpl] (Work-Job-Executor-28:[ctx-5b29434d, job-113/job-141, ctx-9e239170]) (logid:cbf99996) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create volume [{"name":"ROOT-38","uuid":"32eba291-1115-4dfe-b90c-a64de46a2de4"}] due to [com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: internal error: Child process (/usr/bin/mount -o nodev,nosuid,noexec 10.100.7.4:/secondary/template/tmpl/1/3 /mnt/3571e6e5-e631-3d46-8a7e-b2b66f8f4bc4) unexpected exit status 32: mount.nfs: Protocol not supported
].
        at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume(VolumeOrchestrator.java:1797)
        at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1944)
        at jdk.internal.reflect.GeneratedMethodAccessor221.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
        at jdk.proxy3/jdk.proxy3.$Proxy265.prepare(Unknown Source)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1265)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5467)
        at jdk.internal.reflect.GeneratedMethodAccessor207.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:106)
        at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5591)
        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:652)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:600)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
```

### versions

ACS 4.20
Alma Linux 9
VNXe3200

### The steps to reproduce the bug

1.  
2.
3.
...


### What to do about it?

_No response_

GitHub link: https://github.com/apache/cloudstack/discussions/10555

----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org


Re: [D] Secondry Storage NFS v3 [cloudstack]

Posted by "UAnton (via GitHub)" <gi...@apache.org>.
GitHub user UAnton closed a discussion: Secondry Storage NFS v3

### problem

KVM nodes can't deploy Service VMs.

```
2025-03-11 21:32:30,235 ERROR [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-28:[ctx-5b29434d, job-113/job-141, ctx-9e239170]) (logid:cbf99996) Unable to create volume [{"name":"ROOT-38","uuid":"32eba291-1115-4dfe-b90c-a64de46a2de4"}] due to [com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: internal error: Child process (/usr/bin/mount -o nodev,nosuid,noexec 10.100.7.4:/secondary/template/tmpl/1/3 /mnt/3571e6e5-e631-3d46-8a7e-b2b66f8f4bc4) unexpected exit status 32: mount.nfs: Protocol not supported
].
2025-03-11 21:32:30,235 WARN  [c.c.v.ClusteredVirtualMachineManagerImpl] (Work-Job-Executor-28:[ctx-5b29434d, job-113/job-141, ctx-9e239170]) (logid:cbf99996) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create volume [{"name":"ROOT-38","uuid":"32eba291-1115-4dfe-b90c-a64de46a2de4"}] due to [com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: internal error: Child process (/usr/bin/mount -o nodev,nosuid,noexec 10.100.7.4:/secondary/template/tmpl/1/3 /mnt/3571e6e5-e631-3d46-8a7e-b2b66f8f4bc4) unexpected exit status 32: mount.nfs: Protocol not supported
].
        at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume(VolumeOrchestrator.java:1797)
        at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1944)
        at jdk.internal.reflect.GeneratedMethodAccessor221.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
        at jdk.proxy3/jdk.proxy3.$Proxy265.prepare(Unknown Source)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1265)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5467)
        at jdk.internal.reflect.GeneratedMethodAccessor207.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:106)
        at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5591)
        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:652)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:600)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
```

### versions

ACS 4.20
Alma Linux 9
VNXe3200

### The steps to reproduce the bug

1.  
2.
3.
...


### What to do about it?

_No response_

GitHub link: https://github.com/apache/cloudstack/discussions/10555

----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org


Re: [D] Secondry Storage NFS v3 [cloudstack]

Posted by "nischalnischal2020 (via GitHub)" <gi...@apache.org>.
GitHub user nischalnischal2020 added a comment to the discussion: Secondry Storage NFS v3

Hi @UAnton 

Can you please manually mount the nfs share on the KVM hosts and check if it works?

GitHub link: https://github.com/apache/cloudstack/discussions/10555#discussioncomment-12473767

----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org


Re: [D] Secondry Storage NFS v3 [cloudstack]

Posted by "nischalnischal2020 (via GitHub)" <gi...@apache.org>.
GitHub user nischalnischal2020 added a comment to the discussion: Secondry Storage NFS v3

HI @UAnton ,

This error occurs when there is a mismatch in the NFS versions that cloudstack is trying to mount and the version that you are using on the nfs server.
 

Please check the nfs version mentioned in your Configuration  "secstorage.nfs.version"

Also check if the nfs server is running on a higher version only 

you can check the version of nfs on the server using

rpcinfo -p <server-ip>

Please make sure that version matches in cloudstack and nfs server





GitHub link: https://github.com/apache/cloudstack/discussions/10555#discussioncomment-12473765

----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org


Re: [D] Secondry Storage NFS v3 [cloudstack]

Posted by "UAnton (via GitHub)" <gi...@apache.org>.
GitHub user UAnton added a comment to the discussion: Secondry Storage NFS v3

Hi @nischalnischal2020,
Yes, share is work but with **-o vers=3**

GitHub link: https://github.com/apache/cloudstack/discussions/10555#discussioncomment-12475667

----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org


Re: [D] Secondry Storage NFS v3 [cloudstack]

Posted by "UAnton (via GitHub)" <gi...@apache.org>.
GitHub user UAnton added a comment to the discussion: Secondry Storage NFS v3

@nischalnischal2020 Hi,
nfs-server version is 3.

secstorage.nfs.version
<img width="2006" alt="Image" src="https://github.com/user-attachments/assets/dbd2b01d-5d1d-46f4-9f65-bd76cd187825" />

GitHub link: https://github.com/apache/cloudstack/discussions/10555#discussioncomment-12473766

----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org


Re: [D] Secondry Storage NFS v3 [cloudstack]

Posted by "UAnton (via GitHub)" <gi...@apache.org>.
GitHub user UAnton closed the discussion with a comment: Secondry Storage NFS v3

I found a solution how to enable v4 on my old VNXe3200.

GitHub link: https://github.com/apache/cloudstack/discussions/10555#discussioncomment-12478526

----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org