You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "midhunpjos (via GitHub)" <gi...@apache.org> on 2025/02/06 10:09:00 UTC
[I] Cloudstack Agent Java IO exception [cloudstack]
midhunpjos opened a new issue, #10334:
URL: https://github.com/apache/cloudstack/issues/10334
### problem
Network Actions,VM Deployments,router health checks are failing.
On cloudstack agent logs below is printed
2025-02-06 09:14:09,343 WARN [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:[]) (logid:64df14bf) Exception [Cannot run program "/usr/share/
cloudstack-common/scripts/network/domr/router_proxy.sh": error=0, Failed to exec spawn helper: pid: 605941, exit value: 1] occurred when attempting to run command [/usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh filesystem_writable_check.py 169.254.156.203 ]. java.io.IOException: Cannot run prog
ram "/usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh": error=0, Failed to exec spawn helper: pid: 605941, exit value: 1
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
at com.cloud.utils.script.Script.execute(Script.java:232)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeInVR(LibvirtComputingResource.java:553)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeInVR(LibvirtComputingResource.java:541)
at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.checkRouterFileSystem(VirtualRoutingResource.java:419)
at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.execute(VirtualRoutingResource.java:391)
at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeQueryCommand(VirtualRoutingResource.java:227)
at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeRequest(VirtualRoutingResource.java:129)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtNetworkElementCommandWrapper.execute(LibvirtNetworkElementCommandWrapper.java:35)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtNetworkElementCommandWrapper.execute(LibvirtNetworkElementCommandWrapper.java:29)
at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)
at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1945)
at com.cloud.agent.Agent.processRequest(Agent.java:686)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1109)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.io.IOException: error=0, Failed to exec spawn helper: pid: 605941, exit value: 1
at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
... 13 more
2025-02-06 09:14:46,697 WARN [kvm.storage.MultipathSCSIAdapterBase] (MultipathMapCleanupJob:[]) (logid:) Exception [Cannot run program "/usr/share/cloudstac
k-common/scripts/storage/multipath/cleanStaleMaps.sh": error=0, Failed to exec spawn helper: pid: 605943, exit value: 1] occurred when attempting to run command [/usr/share/cloudstack-common/scripts/storage/multipath/cleanStaleMaps.sh ]. java.io.IOException: Cannot run program "/usr/share/cloudstack-common/script
s/storage/multipath/cleanStaleMaps.sh": error=0, Failed to exec spawn helper: pid: 605943, exit value: 1
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
at com.cloud.utils.script.Script.execute(Script.java:232)
at com.cloud.hypervisor.kvm.storage.MultipathSCSIAdapterBase.runScript(MultipathSCSIAdapterBase.java:382)
at com.cloud.hypervisor.kvm.storage.MultipathSCSIAdapterBase.cleanupStaleMaps(MultipathSCSIAdapterBase.java:572)
at com.cloud.hypervisor.kvm.storage.MultipathSCSIAdapterBase$1.run(MultipathSCSIAdapterBase.java:122)
at java.base/java.util.TimerThread.mainLoop(Timer.java:566)
at java.base/java.util.TimerThread.run(Timer.java:516)
Caused by: java.io.IOException: error=0, Failed to exec spawn helper: pid: 605943, exit value: 1
at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
... 7 more
### versions
Cloudstack 4.20 + KVM
CEPH as Primary Storage
### The steps to reproduce the bug
1.Deploy Instance, add firewall rules for guest networks are failing
2.
3.
...
### What to do about it?
Restarting Cloudstack agent resolved the issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cloudstack Agent Java IO exception [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #10334:
URL: https://github.com/apache/cloudstack/issues/10334#issuecomment-2639473857
@midhunpjos , it looks like the agent is not allowed to write on disk. If so only restarting the agent would not solve the issue on NFS. I do not know what the behaviour on Ceph is. Can you reliably reproduce this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cloudstack Agent Java IO exception [cloudstack]
Posted by "midhunpjos (via GitHub)" <gi...@apache.org>.
midhunpjos commented on issue #10334:
URL: https://github.com/apache/cloudstack/issues/10334#issuecomment-2639524766
@DaanHoogland It happened to all Hosts in a cluster and just restarting agent everything was back to normal in all hosts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cloudstack Agent Java IO exception [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland closed issue #10334: Cloudstack Agent Java IO exception
URL: https://github.com/apache/cloudstack/issues/10334
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org