Skip to content

Secondry Storage NFS v3 #10548

Closed
Closed

Description

@UAnton

problem

KVM nodes can't deploy Service VMs.

2025-03-11 21:32:30,235 ERROR [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-28:[ctx-5b29434d, job-113/job-141, ctx-9e239170]) (logid:cbf99996) Unable to create volume [{"name":"ROOT-38","uuid":"32eba291-1115-4dfe-b90c-a64de46a2de4"}] due to [com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: internal error: Child process (/usr/bin/mount -o nodev,nosuid,noexec 10.100.7.4:/secondary/template/tmpl/1/3 /mnt/3571e6e5-e631-3d46-8a7e-b2b66f8f4bc4) unexpected exit status 32: mount.nfs: Protocol not supported
].
2025-03-11 21:32:30,235 WARN  [c.c.v.ClusteredVirtualMachineManagerImpl] (Work-Job-Executor-28:[ctx-5b29434d, job-113/job-141, ctx-9e239170]) (logid:cbf99996) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create volume [{"name":"ROOT-38","uuid":"32eba291-1115-4dfe-b90c-a64de46a2de4"}] due to [com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: internal error: Child process (/usr/bin/mount -o nodev,nosuid,noexec 10.100.7.4:/secondary/template/tmpl/1/3 /mnt/3571e6e5-e631-3d46-8a7e-b2b66f8f4bc4) unexpected exit status 32: mount.nfs: Protocol not supported
].
        at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume(VolumeOrchestrator.java:1797)
        at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(VolumeOrchestrator.java:1944)
        at jdk.internal.reflect.GeneratedMethodAccessor221.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
        at jdk.proxy3/jdk.proxy3.$Proxy265.prepare(Unknown Source)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1265)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5467)
        at jdk.internal.reflect.GeneratedMethodAccessor207.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:106)
        at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5591)
        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:652)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:600)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)

versions

ACS 4.20
Alma Linux 9
VNXe3200

The steps to reproduce the bug

...

What to do about it?

No response

Activity

  1. nischalnischal2020 commented on Mar 12, 2025

    @nischalnischal2020

    HI @UAnton ,

    This error occurs when there is a mismatch in the NFS versions that cloudstack is trying to mount and the version that you are using on the nfs server.

    Please check the nfs version mentioned in your Configuration "secstorage.nfs.version"

    Also check if the nfs server is running on a higher version only

    you can check the version of nfs on the server using

    rpcinfo -p

    Please make sure that version matches in cloudstack and nfs server

  2. UAnton commented on Mar 12, 2025

    @UAnton
    Author

    @nischalnischal2020 Hi,
    nfs-server version is 3.

    secstorage.nfs.version
    Image

  3. nischalnischal2020 commented on Mar 12, 2025

    @nischalnischal2020

    Hi @UAnton

    Can you please manually mount the nfs share on the KVM hosts and check if it works?

  4. locked and limited conversation to collaborators on Mar 12, 2025
  5. converted this issue into a discussion #10555 on Mar 12, 2025
  6. added this to the unplanned milestone on Sep 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions