You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Tejas Gadaria <re...@gmail.com> on 2015/01/05 15:25:40 UTC
Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount error
[opterr=mount failed with return code 32])
Hi,
I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
While creating Zone with NFS as primary & secondary storage
2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing request
2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-2:ctx-23cfb66b) Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6133)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:5074)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:475)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61)
at
com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
return code 32]
at com.xensource.xenapi.Types.checkResponse(Types.java:2006)
at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
at com.xensource.xenapi.SR.create(SR.java:801)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6129)
... 17 more
2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: { Ans:
, MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10, {
Answer } }
2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing class
com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
2015-01-05 19:29:47,126 WARN
[o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
through host 1 due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store: Can
not create storage pool through host 1 due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
com.cloud.utils.exception.CloudRuntimeException: Can not create storage
pool through host 1 due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
at
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImpl.java:384)
at
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:407)
at
com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:669)
at
com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:178)
at
org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:163)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95 --
GET
command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&podId=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyRSPltz6ETNUkeE%3D&_=1420466385645
2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no Primary
Storage UPintenance mode
Same time Xenserver host showing following log..
xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ; NFS
mount error [opterr=mount failed with return code 32]; ]
Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:17 xenserver-01 last message repeated 2 times
Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or
NFS mount defined in /opt/cloud/bin/heartbeat!
Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or
NFS mount defined in /opt/cloud/bin/heartbeat!
Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
Tried to stop iptables on XS host
I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
Need help on this..
Regards,
Tejas
Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Prashant Kumar Mishra <pr...@citrix.com>.
1. Make sure you have 'no_root_squash' in your /etc/exports
2. Make sure you have your portmap listening on all interface.
3. Make sure you have NFS4 specifically turned off.
4. Lastly, make sure iptables allows NFS related ports. You can disable
the firewall for time being until you can add the NFS SR.
Hope this help.
On 1/6/15, 9:40 PM, "Somesh Naidu" <So...@citrix.com> wrote:
>Can you try with *async* instead?
>
>If it still fails, check SMlog to see if there are more details why the
>mount is failing when creating the SR.
>
>-----Original Message-----
>From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
>Sent: Tuesday, January 06, 2015 7:44 AM
>To: dev@cloudstack.apache.org
>Cc: users@cloudstack.apache.org
>Subject: Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
>mount error [opterr=mount failed with return code 32])
>
>Hi Prashant,
>
>/etc/export file content is given below
>/mnt/data/VM-TT/primary *(rw,sync,no_root_squash,no_subtree_check)
>
>Still problem persist.
>
>I have Ubuntu 12.04 Server acting as NFS-Server.
>
>Regards,
>Tejas
>
>
>On Tue, Jan 6, 2015 at 12:40 PM, Prashant Kumar Mishra <
>prashantkumar.mishra@citrix.com> wrote:
>
>> It seems issue with remote mounting ,similar issue seen
>> http://lists.xenproject.org/archives/html/xen-api/2012-05/msg00191.html
>> ,mentioned solution should work in your case also
>>
>> Thanks
>> prashant
>>
>> On 1/6/15, 11:39 AM, "Tejas Gadaria" <re...@gmail.com> wrote:
>>
>> >Hi Suresh,
>> >
>> >I was able to mount NFS on XS host through CLI mount.nfs4 successfully.
>> >Also vhd-utils was present on Management server and XS host also. Still
>> >facing same issue.
>> >
>> >Regards,
>> >Tejas
>> >
>> >
>> >
>> >On Mon, Jan 5, 2015 at 11:16 PM, Suresh Sadhu <Su...@citrix.com>
>> >wrote:
>> >
>> >> Hope you have already copied vhd-util in to the host and also as
>> >>Somesh
>> >> suggested try manually mounting the storage directly from the host.
>> >>
>> >>
>> >> Regards
>> >> Sadhu
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
>> >> Sent: 05 January 2015 20:16
>> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
>> >> Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
>> >>mount
>> >> error [opterr=mount failed with return code 32])
>> >>
>> >> XS is unable to create the SR failing with the following error:
>> >> ---
>> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
>> >>with
>> >> return code 32]
>> >> ---
>> >>
>> >> May be a problem with the NFS. Can you try manually creating a SR on
>>XS
>> >> using this NFS or may be simply try to mount the NFS on XS and see if
>> >>that
>> >> works.
>> >>
>> >> -----Original Message-----
>> >> From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
>> >> Sent: Monday, January 05, 2015 9:26 AM
>> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
>> >> Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
>>mount
>> >> error [opterr=mount failed with return code 32])
>> >>
>> >> Hi,
>> >>
>> >> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
>> >>
>> >>
>> >> While creating Zone with NFS as primary & secondary storage
>> >>
>> >> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
>> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing
>> >>request
>> >> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
>> >> (DirectAgent-2:ctx-23cfb66b) Catch Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary
>> >> com.cloud.utils.exception.CloudRuntimeException: Unable to create
>>NFS SR
>> >> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixRes
>>>>ou
>> >>rceBase.java:6133)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixReso
>>>>ur
>> >>ceBase.java:5074)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(Cit
>>>>ri
>> >>xResourceBase.java:475)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(Xe
>>>>nS
>> >>erver56Resource.java:61)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(X
>>>>en
>> >>Server610Resource.java:102)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgen
>>>>tA
>> >>ttache.java:307)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Mana
>>>>ge
>> >>dContextRunnable.java:49)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call
>>>>(D
>> >>efaultManagedContext.java:56)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWi
>>>>th
>> >>Context(DefaultManagedContext.java:103)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWit
>>>>hC
>> >>ontext(DefaultManagedContext.java:53)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manage
>>>>dC
>> >>ontextRunnable.java:46)
>> >> at
>>
>>>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ac
>>>>ce
>> >>ss$201(ScheduledThreadPoolExecutor.java:178)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ru
>>>>n(
>> >>ScheduledThreadPoolExecutor.java:292)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja
>>>>va
>> >>:1145)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>>>av
>> >>a:615)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
>> >>with
>> >> return code 32] at
>> >>com.xensource.xenapi.Types.checkResponse(Types.java:2006)
>> >> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConn
>>>>ec
>> >>tion.dispatch(XenServerConnectionPool.java:456)
>> >> at com.xensource.xenapi.SR.create(SR.java:801)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixRes
>>>>ou
>> >>rceBase.java:6129)
>> >> ... 17 more
>> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
>> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
>> >>Received:
>> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
>> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
>> >>Received:
>> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
>> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing:
>>{
>> >>Ans:
>> >> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
>> >> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch
>> >>Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
>> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq
>>1-3875347479352311816:
>> >> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags:
>> >>10, {
>> >> Answer } }
>> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing
>> >>class
>> >> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary
>> >> 2015-01-05 19:29:47,126 WARN
>> >> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage
>>pool
>> >> through host 1 due to Catch Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary
>> >> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data
>>store:
>> >>Can
>> >> not create storage pool through host 1 due to Catch Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary
>> >> com.cloud.utils.exception.CloudRuntimeException: Can not create
>>storage
>> >> pool through host 1 due to Catch Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryData
>>>>St
>>
>>>>oreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleI
>>>>mp
>> >>l.java:384)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryData
>>>>St
>>
>>>>oreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.
>>>>ja
>> >>va:407)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java
>>>>:6
>> >>69)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java
>>>>:1
>> >>78)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.ex
>>>>ec
>> >>ute(CreateStoragePoolCmd.java:163)
>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
>> >> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
>> >> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
>> >> at
>>com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
>> >> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
>> >> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call
>>>>(D
>> >>efaultManagedContext.java:56)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWi
>>>>th
>> >>Context(DefaultManagedContext.java:103)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWit
>>>>hC
>> >>ontext(DefaultManagedContext.java:53)
>> >> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
>> >> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
>> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
>>>>at
>> >>ionFilterChain.java:290)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
>>>>te
>> >>rChain.java:206)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperVal
>>>>ve
>> >>.java:233)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.StandardContextValve.invoke(StandardContextVal
>>>>ve
>> >>.java:191)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.jav
>>>>a:
>> >>127)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.jav
>>>>a:
>> >>102)
>> >> at
>> >>
>>
>>>>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:55
>>>>5)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve
>>>>.j
>> >>ava:109)
>> >> at
>> >>
>>
>>>>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
>>>>29
>> >>8)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.
>>>>ja
>> >>va:889)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.proc
>>>>es
>> >>s(Http11NioProtocol.java:721)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.
>>>>ja
>> >>va:2274)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja
>>>>va
>> >>:1145)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>>>av
>> >>a:615)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
>> >> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END===
>>192.168.160.95
>> >>--
>> >> GET
>> >>
>> >>
>>
>>>>command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&p
>>>>od
>>
>>>>Id=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-91
>>>>58
>>
>>>>-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.11
>>>>0%
>>
>>>>2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKO
>>>>yR
>> >>SPltz6ETNUkeE%3D&_=1420466385645
>> >> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> >> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
>> >> Primary Storage UPintenance mode
>> >>
>> >>
>> >> Same time Xenserver host showing following log..
>> >>
>> >>
>> >>
>> >> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
>> >> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [
>>;
>> >>NFS
>> >> mount error [opterr=mount failed with return code 32]; ] Jan 5
>> >>19:29:47
>> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
>> >> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5
>> >>19:30:17
>> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
>> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no
>>iSCSI
>> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
>> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no
>>iSCSI
>> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
>> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
>> >>
>> >>
>> >> Tried to stop iptables on XS host
>> >> I checked /opt/cloud/bin/heartbeat file, on XS host and file is
>>empty.
>> >>
>> >>
>> >> Need help on this..
>> >>
>> >>
>> >>
>> >> Regards,
>> >> Tejas
>> >>
>>
>>
Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Prashant Kumar Mishra <pr...@citrix.com>.
1. Make sure you have 'no_root_squash' in your /etc/exports
2. Make sure you have your portmap listening on all interface.
3. Make sure you have NFS4 specifically turned off.
4. Lastly, make sure iptables allows NFS related ports. You can disable
the firewall for time being until you can add the NFS SR.
Hope this help.
On 1/6/15, 9:40 PM, "Somesh Naidu" <So...@citrix.com> wrote:
>Can you try with *async* instead?
>
>If it still fails, check SMlog to see if there are more details why the
>mount is failing when creating the SR.
>
>-----Original Message-----
>From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
>Sent: Tuesday, January 06, 2015 7:44 AM
>To: dev@cloudstack.apache.org
>Cc: users@cloudstack.apache.org
>Subject: Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
>mount error [opterr=mount failed with return code 32])
>
>Hi Prashant,
>
>/etc/export file content is given below
>/mnt/data/VM-TT/primary *(rw,sync,no_root_squash,no_subtree_check)
>
>Still problem persist.
>
>I have Ubuntu 12.04 Server acting as NFS-Server.
>
>Regards,
>Tejas
>
>
>On Tue, Jan 6, 2015 at 12:40 PM, Prashant Kumar Mishra <
>prashantkumar.mishra@citrix.com> wrote:
>
>> It seems issue with remote mounting ,similar issue seen
>> http://lists.xenproject.org/archives/html/xen-api/2012-05/msg00191.html
>> ,mentioned solution should work in your case also
>>
>> Thanks
>> prashant
>>
>> On 1/6/15, 11:39 AM, "Tejas Gadaria" <re...@gmail.com> wrote:
>>
>> >Hi Suresh,
>> >
>> >I was able to mount NFS on XS host through CLI mount.nfs4 successfully.
>> >Also vhd-utils was present on Management server and XS host also. Still
>> >facing same issue.
>> >
>> >Regards,
>> >Tejas
>> >
>> >
>> >
>> >On Mon, Jan 5, 2015 at 11:16 PM, Suresh Sadhu <Su...@citrix.com>
>> >wrote:
>> >
>> >> Hope you have already copied vhd-util in to the host and also as
>> >>Somesh
>> >> suggested try manually mounting the storage directly from the host.
>> >>
>> >>
>> >> Regards
>> >> Sadhu
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
>> >> Sent: 05 January 2015 20:16
>> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
>> >> Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
>> >>mount
>> >> error [opterr=mount failed with return code 32])
>> >>
>> >> XS is unable to create the SR failing with the following error:
>> >> ---
>> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
>> >>with
>> >> return code 32]
>> >> ---
>> >>
>> >> May be a problem with the NFS. Can you try manually creating a SR on
>>XS
>> >> using this NFS or may be simply try to mount the NFS on XS and see if
>> >>that
>> >> works.
>> >>
>> >> -----Original Message-----
>> >> From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
>> >> Sent: Monday, January 05, 2015 9:26 AM
>> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
>> >> Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
>>mount
>> >> error [opterr=mount failed with return code 32])
>> >>
>> >> Hi,
>> >>
>> >> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
>> >>
>> >>
>> >> While creating Zone with NFS as primary & secondary storage
>> >>
>> >> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
>> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing
>> >>request
>> >> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
>> >> (DirectAgent-2:ctx-23cfb66b) Catch Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary
>> >> com.cloud.utils.exception.CloudRuntimeException: Unable to create
>>NFS SR
>> >> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixRes
>>>>ou
>> >>rceBase.java:6133)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixReso
>>>>ur
>> >>ceBase.java:5074)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(Cit
>>>>ri
>> >>xResourceBase.java:475)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(Xe
>>>>nS
>> >>erver56Resource.java:61)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(X
>>>>en
>> >>Server610Resource.java:102)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgen
>>>>tA
>> >>ttache.java:307)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Mana
>>>>ge
>> >>dContextRunnable.java:49)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call
>>>>(D
>> >>efaultManagedContext.java:56)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWi
>>>>th
>> >>Context(DefaultManagedContext.java:103)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWit
>>>>hC
>> >>ontext(DefaultManagedContext.java:53)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manage
>>>>dC
>> >>ontextRunnable.java:46)
>> >> at
>>
>>>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ac
>>>>ce
>> >>ss$201(ScheduledThreadPoolExecutor.java:178)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ru
>>>>n(
>> >>ScheduledThreadPoolExecutor.java:292)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja
>>>>va
>> >>:1145)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>>>av
>> >>a:615)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
>> >>with
>> >> return code 32] at
>> >>com.xensource.xenapi.Types.checkResponse(Types.java:2006)
>> >> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConn
>>>>ec
>> >>tion.dispatch(XenServerConnectionPool.java:456)
>> >> at com.xensource.xenapi.SR.create(SR.java:801)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixRes
>>>>ou
>> >>rceBase.java:6129)
>> >> ... 17 more
>> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
>> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
>> >>Received:
>> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
>> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
>> >>Received:
>> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
>> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing:
>>{
>> >>Ans:
>> >> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
>> >> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch
>> >>Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
>> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq
>>1-3875347479352311816:
>> >> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags:
>> >>10, {
>> >> Answer } }
>> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing
>> >>class
>> >> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary
>> >> 2015-01-05 19:29:47,126 WARN
>> >> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage
>>pool
>> >> through host 1 due to Catch Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary
>> >> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data
>>store:
>> >>Can
>> >> not create storage pool through host 1 due to Catch Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary
>> >> com.cloud.utils.exception.CloudRuntimeException: Can not create
>>storage
>> >> pool through host 1 due to Catch Exception
>> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>> >>failed
>> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
>>create
>> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> >> 192.168.161.110/mnt/data/VMS-TT/primary
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryData
>>>>St
>>
>>>>oreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleI
>>>>mp
>> >>l.java:384)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryData
>>>>St
>>
>>>>oreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.
>>>>ja
>> >>va:407)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java
>>>>:6
>> >>69)
>> >> at
>> >>
>> >>
>>
>>>>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java
>>>>:1
>> >>78)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.ex
>>>>ec
>> >>ute(CreateStoragePoolCmd.java:163)
>> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
>> >> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
>> >> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
>> >> at
>>com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
>> >> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
>> >> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call
>>>>(D
>> >>efaultManagedContext.java:56)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWi
>>>>th
>> >>Context(DefaultManagedContext.java:103)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWit
>>>>hC
>> >>ontext(DefaultManagedContext.java:53)
>> >> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
>> >> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
>> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applic
>>>>at
>> >>ionFilterChain.java:290)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFil
>>>>te
>> >>rChain.java:206)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperVal
>>>>ve
>> >>.java:233)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.StandardContextValve.invoke(StandardContextVal
>>>>ve
>> >>.java:191)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.jav
>>>>a:
>> >>127)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.jav
>>>>a:
>> >>102)
>> >> at
>> >>
>>
>>>>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:55
>>>>5)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve
>>>>.j
>> >>ava:109)
>> >> at
>> >>
>>
>>>>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
>>>>29
>> >>8)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.
>>>>ja
>> >>va:889)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.proc
>>>>es
>> >>s(Http11NioProtocol.java:721)
>> >> at
>> >>
>> >>
>>
>>>>org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.
>>>>ja
>> >>va:2274)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja
>>>>va
>> >>:1145)
>> >> at
>> >>
>> >>
>>
>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j
>>>>av
>> >>a:615)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
>> >> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
>> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END===
>>192.168.160.95
>> >>--
>> >> GET
>> >>
>> >>
>>
>>>>command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&p
>>>>od
>>
>>>>Id=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-91
>>>>58
>>
>>>>-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.11
>>>>0%
>>
>>>>2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKO
>>>>yR
>> >>SPltz6ETNUkeE%3D&_=1420466385645
>> >> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> >> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
>> >> Primary Storage UPintenance mode
>> >>
>> >>
>> >> Same time Xenserver host showing following log..
>> >>
>> >>
>> >>
>> >> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
>> >> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [
>>;
>> >>NFS
>> >> mount error [opterr=mount failed with return code 32]; ] Jan 5
>> >>19:29:47
>> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
>> >> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5
>> >>19:30:17
>> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
>> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no
>>iSCSI
>> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
>> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no
>>iSCSI
>> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
>> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
>> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>> >>0.0.0.0:80|event.from
>> >> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
>> >>
>> >>
>> >> Tried to stop iptables on XS host
>> >> I checked /opt/cloud/bin/heartbeat file, on XS host and file is
>>empty.
>> >>
>> >>
>> >> Need help on this..
>> >>
>> >>
>> >>
>> >> Regards,
>> >> Tejas
>> >>
>>
>>
RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Somesh Naidu <So...@citrix.com>.
Can you try with *async* instead?
If it still fails, check SMlog to see if there are more details why the mount is failing when creating the SR.
-----Original Message-----
From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
Sent: Tuesday, January 06, 2015 7:44 AM
To: dev@cloudstack.apache.org
Cc: users@cloudstack.apache.org
Subject: Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32])
Hi Prashant,
/etc/export file content is given below
/mnt/data/VM-TT/primary *(rw,sync,no_root_squash,no_subtree_check)
Still problem persist.
I have Ubuntu 12.04 Server acting as NFS-Server.
Regards,
Tejas
On Tue, Jan 6, 2015 at 12:40 PM, Prashant Kumar Mishra <
prashantkumar.mishra@citrix.com> wrote:
> It seems issue with remote mounting ,similar issue seen
> http://lists.xenproject.org/archives/html/xen-api/2012-05/msg00191.html
> ,mentioned solution should work in your case also
>
> Thanks
> prashant
>
> On 1/6/15, 11:39 AM, "Tejas Gadaria" <re...@gmail.com> wrote:
>
> >Hi Suresh,
> >
> >I was able to mount NFS on XS host through CLI mount.nfs4 successfully.
> >Also vhd-utils was present on Management server and XS host also. Still
> >facing same issue.
> >
> >Regards,
> >Tejas
> >
> >
> >
> >On Mon, Jan 5, 2015 at 11:16 PM, Suresh Sadhu <Su...@citrix.com>
> >wrote:
> >
> >> Hope you have already copied vhd-util in to the host and also as
> >>Somesh
> >> suggested try manually mounting the storage directly from the host.
> >>
> >>
> >> Regards
> >> Sadhu
> >>
> >>
> >> -----Original Message-----
> >> From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
> >> Sent: 05 January 2015 20:16
> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> >> Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
> >>mount
> >> error [opterr=mount failed with return code 32])
> >>
> >> XS is unable to create the SR failing with the following error:
> >> ---
> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
> >>with
> >> return code 32]
> >> ---
> >>
> >> May be a problem with the NFS. Can you try manually creating a SR on XS
> >> using this NFS or may be simply try to mount the NFS on XS and see if
> >>that
> >> works.
> >>
> >> -----Original Message-----
> >> From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
> >> Sent: Monday, January 05, 2015 9:26 AM
> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> >> Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
> >> error [opterr=mount failed with return code 32])
> >>
> >> Hi,
> >>
> >> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
> >>
> >>
> >> While creating Zone with NFS as primary & secondary storage
> >>
> >> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing
> >>request
> >> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
> >> (DirectAgent-2:ctx-23cfb66b) Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
> >> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
> >>rceBase.java:6133)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResour
> >>ceBase.java:5074)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(Citri
> >>xResourceBase.java:475)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenS
> >>erver56Resource.java:61)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(Xen
> >>Server610Resource.java:102)
> >> at
> >>
> >>
> >>com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentA
> >>ttache.java:307)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Manage
> >>dContextRunnable.java:49)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
> >>efaultManagedContext.java:56)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
> >>Context(DefaultManagedContext.java:103)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
> >>ontext(DefaultManagedContext.java:53)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedC
> >>ontextRunnable.java:46)
> >> at
> >>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> >> at
> >>
> >>
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acce
> >>ss$201(ScheduledThreadPoolExecutor.java:178)
> >> at
> >>
> >>
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
> >>ScheduledThreadPoolExecutor.java:292)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
> >>:1145)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
> >>a:615)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
> >>with
> >> return code 32] at
> >>com.xensource.xenapi.Types.checkResponse(Types.java:2006)
> >> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnec
> >>tion.dispatch(XenServerConnectionPool.java:456)
> >> at com.xensource.xenapi.SR.create(SR.java:801)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
> >>rceBase.java:6129)
> >> ... 17 more
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
> >>Received:
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
> >>Received:
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: {
> >>Ans:
> >> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
> >> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch
> >>Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
> >> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags:
> >>10, {
> >> Answer } }
> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing
> >>class
> >> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> 2015-01-05 19:29:47,126 WARN
> >> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
> >> through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store:
> >>Can
> >> not create storage pool through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> com.cloud.utils.exception.CloudRuntimeException: Can not create storage
> >> pool through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> at
> >>
> >>
> >>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
> >>oreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImp
> >>l.java:384)
> >> at
> >>
> >>
> >>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
> >>oreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.ja
> >>va:407)
> >> at
> >>
> >>
> >>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:6
> >>69)
> >> at
> >>
> >>
> >>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:1
> >>78)
> >> at
> >>
> >>
> >>org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.exec
> >>ute(CreateStoragePoolCmd.java:163)
> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> >> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
> >> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
> >> at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
> >> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
> >> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
> >>efaultManagedContext.java:56)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
> >>Context(DefaultManagedContext.java:103)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
> >>ontext(DefaultManagedContext.java:53)
> >> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
> >> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> >> at
> >>
> >>
> >>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
> >>ionFilterChain.java:290)
> >> at
> >>
> >>
> >>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
> >>rChain.java:206)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
> >>.java:233)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
> >>.java:191)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
> >>127)
> >> at
> >>
> >>
> >>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
> >>102)
> >> at
> >>
> >>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
> >>ava:109)
> >> at
> >>
> >>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:29
> >>8)
> >> at
> >>
> >>
> >>org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.ja
> >>va:889)
> >> at
> >>
> >>
> >>org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.proces
> >>s(Http11NioProtocol.java:721)
> >> at
> >>
> >>
> >>org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.ja
> >>va:2274)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
> >>:1145)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
> >>a:615)
> >> at java.lang.Thread.run(Thread.java:745)
> >> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
> >> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95
> >>--
> >> GET
> >>
> >>
> >>command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&pod
> >>Id=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158
> >>-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%
> >>2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyR
> >>SPltz6ETNUkeE%3D&_=1420466385645
> >> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> >> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
> >> Primary Storage UPintenance mode
> >>
> >>
> >> Same time Xenserver host showing following log..
> >>
> >>
> >>
> >> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
> >> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ;
> >>NFS
> >> mount error [opterr=mount failed with return code 32]; ] Jan 5
> >>19:29:47
> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> >> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5
> >>19:30:17
> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
> >>
> >>
> >> Tried to stop iptables on XS host
> >> I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
> >>
> >>
> >> Need help on this..
> >>
> >>
> >>
> >> Regards,
> >> Tejas
> >>
>
>
RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Somesh Naidu <So...@citrix.com>.
Can you try with *async* instead?
If it still fails, check SMlog to see if there are more details why the mount is failing when creating the SR.
-----Original Message-----
From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
Sent: Tuesday, January 06, 2015 7:44 AM
To: dev@cloudstack.apache.org
Cc: users@cloudstack.apache.org
Subject: Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32])
Hi Prashant,
/etc/export file content is given below
/mnt/data/VM-TT/primary *(rw,sync,no_root_squash,no_subtree_check)
Still problem persist.
I have Ubuntu 12.04 Server acting as NFS-Server.
Regards,
Tejas
On Tue, Jan 6, 2015 at 12:40 PM, Prashant Kumar Mishra <
prashantkumar.mishra@citrix.com> wrote:
> It seems issue with remote mounting ,similar issue seen
> http://lists.xenproject.org/archives/html/xen-api/2012-05/msg00191.html
> ,mentioned solution should work in your case also
>
> Thanks
> prashant
>
> On 1/6/15, 11:39 AM, "Tejas Gadaria" <re...@gmail.com> wrote:
>
> >Hi Suresh,
> >
> >I was able to mount NFS on XS host through CLI mount.nfs4 successfully.
> >Also vhd-utils was present on Management server and XS host also. Still
> >facing same issue.
> >
> >Regards,
> >Tejas
> >
> >
> >
> >On Mon, Jan 5, 2015 at 11:16 PM, Suresh Sadhu <Su...@citrix.com>
> >wrote:
> >
> >> Hope you have already copied vhd-util in to the host and also as
> >>Somesh
> >> suggested try manually mounting the storage directly from the host.
> >>
> >>
> >> Regards
> >> Sadhu
> >>
> >>
> >> -----Original Message-----
> >> From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
> >> Sent: 05 January 2015 20:16
> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> >> Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
> >>mount
> >> error [opterr=mount failed with return code 32])
> >>
> >> XS is unable to create the SR failing with the following error:
> >> ---
> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
> >>with
> >> return code 32]
> >> ---
> >>
> >> May be a problem with the NFS. Can you try manually creating a SR on XS
> >> using this NFS or may be simply try to mount the NFS on XS and see if
> >>that
> >> works.
> >>
> >> -----Original Message-----
> >> From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
> >> Sent: Monday, January 05, 2015 9:26 AM
> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> >> Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
> >> error [opterr=mount failed with return code 32])
> >>
> >> Hi,
> >>
> >> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
> >>
> >>
> >> While creating Zone with NFS as primary & secondary storage
> >>
> >> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing
> >>request
> >> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
> >> (DirectAgent-2:ctx-23cfb66b) Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
> >> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
> >>rceBase.java:6133)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResour
> >>ceBase.java:5074)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(Citri
> >>xResourceBase.java:475)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenS
> >>erver56Resource.java:61)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(Xen
> >>Server610Resource.java:102)
> >> at
> >>
> >>
> >>com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentA
> >>ttache.java:307)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Manage
> >>dContextRunnable.java:49)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
> >>efaultManagedContext.java:56)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
> >>Context(DefaultManagedContext.java:103)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
> >>ontext(DefaultManagedContext.java:53)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedC
> >>ontextRunnable.java:46)
> >> at
> >>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> >> at
> >>
> >>
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acce
> >>ss$201(ScheduledThreadPoolExecutor.java:178)
> >> at
> >>
> >>
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
> >>ScheduledThreadPoolExecutor.java:292)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
> >>:1145)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
> >>a:615)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
> >>with
> >> return code 32] at
> >>com.xensource.xenapi.Types.checkResponse(Types.java:2006)
> >> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnec
> >>tion.dispatch(XenServerConnectionPool.java:456)
> >> at com.xensource.xenapi.SR.create(SR.java:801)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
> >>rceBase.java:6129)
> >> ... 17 more
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
> >>Received:
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
> >>Received:
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: {
> >>Ans:
> >> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
> >> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch
> >>Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
> >> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags:
> >>10, {
> >> Answer } }
> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing
> >>class
> >> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> 2015-01-05 19:29:47,126 WARN
> >> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
> >> through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store:
> >>Can
> >> not create storage pool through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> com.cloud.utils.exception.CloudRuntimeException: Can not create storage
> >> pool through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> at
> >>
> >>
> >>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
> >>oreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImp
> >>l.java:384)
> >> at
> >>
> >>
> >>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
> >>oreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.ja
> >>va:407)
> >> at
> >>
> >>
> >>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:6
> >>69)
> >> at
> >>
> >>
> >>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:1
> >>78)
> >> at
> >>
> >>
> >>org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.exec
> >>ute(CreateStoragePoolCmd.java:163)
> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> >> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
> >> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
> >> at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
> >> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
> >> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
> >>efaultManagedContext.java:56)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
> >>Context(DefaultManagedContext.java:103)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
> >>ontext(DefaultManagedContext.java:53)
> >> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
> >> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> >> at
> >>
> >>
> >>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
> >>ionFilterChain.java:290)
> >> at
> >>
> >>
> >>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
> >>rChain.java:206)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
> >>.java:233)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
> >>.java:191)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
> >>127)
> >> at
> >>
> >>
> >>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
> >>102)
> >> at
> >>
> >>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
> >>ava:109)
> >> at
> >>
> >>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:29
> >>8)
> >> at
> >>
> >>
> >>org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.ja
> >>va:889)
> >> at
> >>
> >>
> >>org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.proces
> >>s(Http11NioProtocol.java:721)
> >> at
> >>
> >>
> >>org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.ja
> >>va:2274)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
> >>:1145)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
> >>a:615)
> >> at java.lang.Thread.run(Thread.java:745)
> >> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
> >> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95
> >>--
> >> GET
> >>
> >>
> >>command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&pod
> >>Id=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158
> >>-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%
> >>2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyR
> >>SPltz6ETNUkeE%3D&_=1420466385645
> >> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> >> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
> >> Primary Storage UPintenance mode
> >>
> >>
> >> Same time Xenserver host showing following log..
> >>
> >>
> >>
> >> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
> >> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ;
> >>NFS
> >> mount error [opterr=mount failed with return code 32]; ] Jan 5
> >>19:29:47
> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> >> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5
> >>19:30:17
> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
> >>
> >>
> >> Tried to stop iptables on XS host
> >> I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
> >>
> >>
> >> Need help on this..
> >>
> >>
> >>
> >> Regards,
> >> Tejas
> >>
>
>
Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Tejas Gadaria <re...@gmail.com>.
Hi Prashant,
/etc/export file content is given below
/mnt/data/VM-TT/primary *(rw,sync,no_root_squash,no_subtree_check)
Still problem persist.
I have Ubuntu 12.04 Server acting as NFS-Server.
Regards,
Tejas
On Tue, Jan 6, 2015 at 12:40 PM, Prashant Kumar Mishra <
prashantkumar.mishra@citrix.com> wrote:
> It seems issue with remote mounting ,similar issue seen
> http://lists.xenproject.org/archives/html/xen-api/2012-05/msg00191.html
> ,mentioned solution should work in your case also
>
> Thanks
> prashant
>
> On 1/6/15, 11:39 AM, "Tejas Gadaria" <re...@gmail.com> wrote:
>
> >Hi Suresh,
> >
> >I was able to mount NFS on XS host through CLI mount.nfs4 successfully.
> >Also vhd-utils was present on Management server and XS host also. Still
> >facing same issue.
> >
> >Regards,
> >Tejas
> >
> >
> >
> >On Mon, Jan 5, 2015 at 11:16 PM, Suresh Sadhu <Su...@citrix.com>
> >wrote:
> >
> >> Hope you have already copied vhd-util in to the host and also as
> >>Somesh
> >> suggested try manually mounting the storage directly from the host.
> >>
> >>
> >> Regards
> >> Sadhu
> >>
> >>
> >> -----Original Message-----
> >> From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
> >> Sent: 05 January 2015 20:16
> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> >> Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
> >>mount
> >> error [opterr=mount failed with return code 32])
> >>
> >> XS is unable to create the SR failing with the following error:
> >> ---
> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
> >>with
> >> return code 32]
> >> ---
> >>
> >> May be a problem with the NFS. Can you try manually creating a SR on XS
> >> using this NFS or may be simply try to mount the NFS on XS and see if
> >>that
> >> works.
> >>
> >> -----Original Message-----
> >> From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
> >> Sent: Monday, January 05, 2015 9:26 AM
> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> >> Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
> >> error [opterr=mount failed with return code 32])
> >>
> >> Hi,
> >>
> >> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
> >>
> >>
> >> While creating Zone with NFS as primary & secondary storage
> >>
> >> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing
> >>request
> >> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
> >> (DirectAgent-2:ctx-23cfb66b) Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
> >> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
> >>rceBase.java:6133)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResour
> >>ceBase.java:5074)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(Citri
> >>xResourceBase.java:475)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenS
> >>erver56Resource.java:61)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(Xen
> >>Server610Resource.java:102)
> >> at
> >>
> >>
> >>com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentA
> >>ttache.java:307)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Manage
> >>dContextRunnable.java:49)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
> >>efaultManagedContext.java:56)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
> >>Context(DefaultManagedContext.java:103)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
> >>ontext(DefaultManagedContext.java:53)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedC
> >>ontextRunnable.java:46)
> >> at
> >>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> >> at
> >>
> >>
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acce
> >>ss$201(ScheduledThreadPoolExecutor.java:178)
> >> at
> >>
> >>
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
> >>ScheduledThreadPoolExecutor.java:292)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
> >>:1145)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
> >>a:615)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
> >>with
> >> return code 32] at
> >>com.xensource.xenapi.Types.checkResponse(Types.java:2006)
> >> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnec
> >>tion.dispatch(XenServerConnectionPool.java:456)
> >> at com.xensource.xenapi.SR.create(SR.java:801)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
> >>rceBase.java:6129)
> >> ... 17 more
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
> >>Received:
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
> >>Received:
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: {
> >>Ans:
> >> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
> >> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch
> >>Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
> >> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags:
> >>10, {
> >> Answer } }
> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing
> >>class
> >> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> 2015-01-05 19:29:47,126 WARN
> >> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
> >> through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store:
> >>Can
> >> not create storage pool through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> com.cloud.utils.exception.CloudRuntimeException: Can not create storage
> >> pool through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> at
> >>
> >>
> >>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
> >>oreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImp
> >>l.java:384)
> >> at
> >>
> >>
> >>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
> >>oreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.ja
> >>va:407)
> >> at
> >>
> >>
> >>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:6
> >>69)
> >> at
> >>
> >>
> >>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:1
> >>78)
> >> at
> >>
> >>
> >>org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.exec
> >>ute(CreateStoragePoolCmd.java:163)
> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> >> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
> >> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
> >> at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
> >> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
> >> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
> >>efaultManagedContext.java:56)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
> >>Context(DefaultManagedContext.java:103)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
> >>ontext(DefaultManagedContext.java:53)
> >> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
> >> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> >> at
> >>
> >>
> >>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
> >>ionFilterChain.java:290)
> >> at
> >>
> >>
> >>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
> >>rChain.java:206)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
> >>.java:233)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
> >>.java:191)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
> >>127)
> >> at
> >>
> >>
> >>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
> >>102)
> >> at
> >>
> >>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
> >>ava:109)
> >> at
> >>
> >>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:29
> >>8)
> >> at
> >>
> >>
> >>org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.ja
> >>va:889)
> >> at
> >>
> >>
> >>org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.proces
> >>s(Http11NioProtocol.java:721)
> >> at
> >>
> >>
> >>org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.ja
> >>va:2274)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
> >>:1145)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
> >>a:615)
> >> at java.lang.Thread.run(Thread.java:745)
> >> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
> >> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95
> >>--
> >> GET
> >>
> >>
> >>command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&pod
> >>Id=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158
> >>-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%
> >>2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyR
> >>SPltz6ETNUkeE%3D&_=1420466385645
> >> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> >> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
> >> Primary Storage UPintenance mode
> >>
> >>
> >> Same time Xenserver host showing following log..
> >>
> >>
> >>
> >> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
> >> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ;
> >>NFS
> >> mount error [opterr=mount failed with return code 32]; ] Jan 5
> >>19:29:47
> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> >> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5
> >>19:30:17
> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
> >>
> >>
> >> Tried to stop iptables on XS host
> >> I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
> >>
> >>
> >> Need help on this..
> >>
> >>
> >>
> >> Regards,
> >> Tejas
> >>
>
>
Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Tejas Gadaria <re...@gmail.com>.
Hi Prashant,
/etc/export file content is given below
/mnt/data/VM-TT/primary *(rw,sync,no_root_squash,no_subtree_check)
Still problem persist.
I have Ubuntu 12.04 Server acting as NFS-Server.
Regards,
Tejas
On Tue, Jan 6, 2015 at 12:40 PM, Prashant Kumar Mishra <
prashantkumar.mishra@citrix.com> wrote:
> It seems issue with remote mounting ,similar issue seen
> http://lists.xenproject.org/archives/html/xen-api/2012-05/msg00191.html
> ,mentioned solution should work in your case also
>
> Thanks
> prashant
>
> On 1/6/15, 11:39 AM, "Tejas Gadaria" <re...@gmail.com> wrote:
>
> >Hi Suresh,
> >
> >I was able to mount NFS on XS host through CLI mount.nfs4 successfully.
> >Also vhd-utils was present on Management server and XS host also. Still
> >facing same issue.
> >
> >Regards,
> >Tejas
> >
> >
> >
> >On Mon, Jan 5, 2015 at 11:16 PM, Suresh Sadhu <Su...@citrix.com>
> >wrote:
> >
> >> Hope you have already copied vhd-util in to the host and also as
> >>Somesh
> >> suggested try manually mounting the storage directly from the host.
> >>
> >>
> >> Regards
> >> Sadhu
> >>
> >>
> >> -----Original Message-----
> >> From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
> >> Sent: 05 January 2015 20:16
> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> >> Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
> >>mount
> >> error [opterr=mount failed with return code 32])
> >>
> >> XS is unable to create the SR failing with the following error:
> >> ---
> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
> >>with
> >> return code 32]
> >> ---
> >>
> >> May be a problem with the NFS. Can you try manually creating a SR on XS
> >> using this NFS or may be simply try to mount the NFS on XS and see if
> >>that
> >> works.
> >>
> >> -----Original Message-----
> >> From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
> >> Sent: Monday, January 05, 2015 9:26 AM
> >> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> >> Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
> >> error [opterr=mount failed with return code 32])
> >>
> >> Hi,
> >>
> >> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
> >>
> >>
> >> While creating Zone with NFS as primary & secondary storage
> >>
> >> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing
> >>request
> >> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
> >> (DirectAgent-2:ctx-23cfb66b) Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
> >> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
> >>rceBase.java:6133)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResour
> >>ceBase.java:5074)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(Citri
> >>xResourceBase.java:475)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenS
> >>erver56Resource.java:61)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(Xen
> >>Server610Resource.java:102)
> >> at
> >>
> >>
> >>com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentA
> >>ttache.java:307)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Manage
> >>dContextRunnable.java:49)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
> >>efaultManagedContext.java:56)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
> >>Context(DefaultManagedContext.java:103)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
> >>ontext(DefaultManagedContext.java:53)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedC
> >>ontextRunnable.java:46)
> >> at
> >>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> >> at
> >>
> >>
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acce
> >>ss$201(ScheduledThreadPoolExecutor.java:178)
> >> at
> >>
> >>
> >>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
> >>ScheduledThreadPoolExecutor.java:292)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
> >>:1145)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
> >>a:615)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
> >>with
> >> return code 32] at
> >>com.xensource.xenapi.Types.checkResponse(Types.java:2006)
> >> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnec
> >>tion.dispatch(XenServerConnectionPool.java:456)
> >> at com.xensource.xenapi.SR.create(SR.java:801)
> >> at
> >>
> >>
> >>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
> >>rceBase.java:6129)
> >> ... 17 more
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
> >>Received:
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
> >>Received:
> >> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
> >> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: {
> >>Ans:
> >> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
> >> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch
> >>Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
> >> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags:
> >>10, {
> >> Answer } }
> >> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing
> >>class
> >> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> 2015-01-05 19:29:47,126 WARN
> >> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
> >> through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store:
> >>Can
> >> not create storage pool through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> com.cloud.utils.exception.CloudRuntimeException: Can not create storage
> >> pool through host 1 due to Catch Exception
> >> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> >>failed
> >> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> >> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> >> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> >> 192.168.161.110/mnt/data/VMS-TT/primary
> >> at
> >>
> >>
> >>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
> >>oreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImp
> >>l.java:384)
> >> at
> >>
> >>
> >>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
> >>oreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.ja
> >>va:407)
> >> at
> >>
> >>
> >>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:6
> >>69)
> >> at
> >>
> >>
> >>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:1
> >>78)
> >> at
> >>
> >>
> >>org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.exec
> >>ute(CreateStoragePoolCmd.java:163)
> >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> >> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
> >> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
> >> at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
> >> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
> >> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
> >>efaultManagedContext.java:56)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
> >>Context(DefaultManagedContext.java:103)
> >> at
> >>
> >>
> >>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
> >>ontext(DefaultManagedContext.java:53)
> >> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
> >> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> >> at
> >>
> >>
> >>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
> >>ionFilterChain.java:290)
> >> at
> >>
> >>
> >>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
> >>rChain.java:206)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
> >>.java:233)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
> >>.java:191)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
> >>127)
> >> at
> >>
> >>
> >>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
> >>102)
> >> at
> >>
> >>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> >> at
> >>
> >>
> >>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
> >>ava:109)
> >> at
> >>
> >>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:29
> >>8)
> >> at
> >>
> >>
> >>org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.ja
> >>va:889)
> >> at
> >>
> >>
> >>org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.proces
> >>s(Http11NioProtocol.java:721)
> >> at
> >>
> >>
> >>org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.ja
> >>va:2274)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
> >>:1145)
> >> at
> >>
> >>
> >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
> >>a:615)
> >> at java.lang.Thread.run(Thread.java:745)
> >> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
> >> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
> >> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95
> >>--
> >> GET
> >>
> >>
> >>command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&pod
> >>Id=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158
> >>-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%
> >>2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyR
> >>SPltz6ETNUkeE%3D&_=1420466385645
> >> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> >> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
> >> Primary Storage UPintenance mode
> >>
> >>
> >> Same time Xenserver host showing following log..
> >>
> >>
> >>
> >> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
> >> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ;
> >>NFS
> >> mount error [opterr=mount failed with return code 32]; ] Jan 5
> >>19:29:47
> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> >> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5
> >>19:30:17
> >> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> >> or NFS mount defined in /opt/cloud/bin/heartbeat!
> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> >> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> >>0.0.0.0:80|event.from
> >> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
> >>
> >>
> >> Tried to stop iptables on XS host
> >> I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
> >>
> >>
> >> Need help on this..
> >>
> >>
> >>
> >> Regards,
> >> Tejas
> >>
>
>
Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Prashant Kumar Mishra <pr...@citrix.com>.
It seems issue with remote mounting ,similar issue seen
http://lists.xenproject.org/archives/html/xen-api/2012-05/msg00191.html
,mentioned solution should work in your case also
Thanks
prashant
On 1/6/15, 11:39 AM, "Tejas Gadaria" <re...@gmail.com> wrote:
>Hi Suresh,
>
>I was able to mount NFS on XS host through CLI mount.nfs4 successfully.
>Also vhd-utils was present on Management server and XS host also. Still
>facing same issue.
>
>Regards,
>Tejas
>
>
>
>On Mon, Jan 5, 2015 at 11:16 PM, Suresh Sadhu <Su...@citrix.com>
>wrote:
>
>> Hope you have already copied vhd-util in to the host and also as
>>Somesh
>> suggested try manually mounting the storage directly from the host.
>>
>>
>> Regards
>> Sadhu
>>
>>
>> -----Original Message-----
>> From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
>> Sent: 05 January 2015 20:16
>> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
>> Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
>>mount
>> error [opterr=mount failed with return code 32])
>>
>> XS is unable to create the SR failing with the following error:
>> ---
>> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
>>with
>> return code 32]
>> ---
>>
>> May be a problem with the NFS. Can you try manually creating a SR on XS
>> using this NFS or may be simply try to mount the NFS on XS and see if
>>that
>> works.
>>
>> -----Original Message-----
>> From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
>> Sent: Monday, January 05, 2015 9:26 AM
>> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
>> Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
>> error [opterr=mount failed with return code 32])
>>
>> Hi,
>>
>> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
>>
>>
>> While creating Zone with NFS as primary & secondary storage
>>
>> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing
>>request
>> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
>> (DirectAgent-2:ctx-23cfb66b) Catch Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary
>> com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
>> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
>>rceBase.java:6133)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResour
>>ceBase.java:5074)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(Citri
>>xResourceBase.java:475)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenS
>>erver56Resource.java:61)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(Xen
>>Server610Resource.java:102)
>> at
>>
>>
>>com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentA
>>ttache.java:307)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Manage
>>dContextRunnable.java:49)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
>>efaultManagedContext.java:56)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
>>Context(DefaultManagedContext.java:103)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
>>ontext(DefaultManagedContext.java:53)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedC
>>ontextRunnable.java:46)
>> at
>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at
>>
>>
>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acce
>>ss$201(ScheduledThreadPoolExecutor.java:178)
>> at
>>
>>
>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
>>ScheduledThreadPoolExecutor.java:292)
>> at
>>
>>
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
>>:1145)
>> at
>>
>>
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
>>a:615)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
>>with
>> return code 32] at
>>com.xensource.xenapi.Types.checkResponse(Types.java:2006)
>> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnec
>>tion.dispatch(XenServerConnectionPool.java:456)
>> at com.xensource.xenapi.SR.create(SR.java:801)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
>>rceBase.java:6129)
>> ... 17 more
>> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
>>Received:
>> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
>>Received:
>> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
>> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: {
>>Ans:
>> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch
>>Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
>> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
>> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags:
>>10, {
>> Answer } }
>> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing
>>class
>> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary
>> 2015-01-05 19:29:47,126 WARN
>> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
>> through host 1 due to Catch Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary
>> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store:
>>Can
>> not create storage pool through host 1 due to Catch Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary
>> com.cloud.utils.exception.CloudRuntimeException: Can not create storage
>> pool through host 1 due to Catch Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary
>> at
>>
>>
>>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
>>oreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImp
>>l.java:384)
>> at
>>
>>
>>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
>>oreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.ja
>>va:407)
>> at
>>
>>
>>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:6
>>69)
>> at
>>
>>
>>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:1
>>78)
>> at
>>
>>
>>org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.exec
>>ute(CreateStoragePoolCmd.java:163)
>> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
>> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
>> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
>> at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
>> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
>> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
>>efaultManagedContext.java:56)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
>>Context(DefaultManagedContext.java:103)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
>>ontext(DefaultManagedContext.java:53)
>> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
>> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>> at
>>
>>
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
>>ionFilterChain.java:290)
>> at
>>
>>
>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
>>rChain.java:206)
>> at
>>
>>
>>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
>>.java:233)
>> at
>>
>>
>>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
>>.java:191)
>> at
>>
>>
>>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
>>127)
>> at
>>
>>
>>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
>>102)
>> at
>>
>>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
>> at
>>
>>
>>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
>>ava:109)
>> at
>>
>>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:29
>>8)
>> at
>>
>>
>>org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.ja
>>va:889)
>> at
>>
>>
>>org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.proces
>>s(Http11NioProtocol.java:721)
>> at
>>
>>
>>org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.ja
>>va:2274)
>> at
>>
>>
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
>>:1145)
>> at
>>
>>
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
>>a:615)
>> at java.lang.Thread.run(Thread.java:745)
>> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
>> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95
>>--
>> GET
>>
>>
>>command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&pod
>>Id=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158
>>-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%
>>2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyR
>>SPltz6ETNUkeE%3D&_=1420466385645
>> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
>> Primary Storage UPintenance mode
>>
>>
>> Same time Xenserver host showing following log..
>>
>>
>>
>> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
>> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ;
>>NFS
>> mount error [opterr=mount failed with return code 32]; ] Jan 5
>>19:29:47
>> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
>> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5
>>19:30:17
>> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
>> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
>> or NFS mount defined in /opt/cloud/bin/heartbeat!
>> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
>> or NFS mount defined in /opt/cloud/bin/heartbeat!
>> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
>>
>>
>> Tried to stop iptables on XS host
>> I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
>>
>>
>> Need help on this..
>>
>>
>>
>> Regards,
>> Tejas
>>
Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Prashant Kumar Mishra <pr...@citrix.com>.
It seems issue with remote mounting ,similar issue seen
http://lists.xenproject.org/archives/html/xen-api/2012-05/msg00191.html
,mentioned solution should work in your case also
Thanks
prashant
On 1/6/15, 11:39 AM, "Tejas Gadaria" <re...@gmail.com> wrote:
>Hi Suresh,
>
>I was able to mount NFS on XS host through CLI mount.nfs4 successfully.
>Also vhd-utils was present on Management server and XS host also. Still
>facing same issue.
>
>Regards,
>Tejas
>
>
>
>On Mon, Jan 5, 2015 at 11:16 PM, Suresh Sadhu <Su...@citrix.com>
>wrote:
>
>> Hope you have already copied vhd-util in to the host and also as
>>Somesh
>> suggested try manually mounting the storage directly from the host.
>>
>>
>> Regards
>> Sadhu
>>
>>
>> -----Original Message-----
>> From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
>> Sent: 05 January 2015 20:16
>> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
>> Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS
>>mount
>> error [opterr=mount failed with return code 32])
>>
>> XS is unable to create the SR failing with the following error:
>> ---
>> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
>>with
>> return code 32]
>> ---
>>
>> May be a problem with the NFS. Can you try manually creating a SR on XS
>> using this NFS or may be simply try to mount the NFS on XS and see if
>>that
>> works.
>>
>> -----Original Message-----
>> From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
>> Sent: Monday, January 05, 2015 9:26 AM
>> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
>> Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
>> error [opterr=mount failed with return code 32])
>>
>> Hi,
>>
>> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
>>
>>
>> While creating Zone with NFS as primary & secondary storage
>>
>> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing
>>request
>> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
>> (DirectAgent-2:ctx-23cfb66b) Catch Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary
>> com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
>> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
>>rceBase.java:6133)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResour
>>ceBase.java:5074)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(Citri
>>xResourceBase.java:475)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenS
>>erver56Resource.java:61)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(Xen
>>Server610Resource.java:102)
>> at
>>
>>
>>com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentA
>>ttache.java:307)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Manage
>>dContextRunnable.java:49)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
>>efaultManagedContext.java:56)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
>>Context(DefaultManagedContext.java:103)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
>>ontext(DefaultManagedContext.java:53)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedC
>>ontextRunnable.java:46)
>> at
>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> at
>>
>>
>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.acce
>>ss$201(ScheduledThreadPoolExecutor.java:178)
>> at
>>
>>
>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
>>ScheduledThreadPoolExecutor.java:292)
>> at
>>
>>
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
>>:1145)
>> at
>>
>>
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
>>a:615)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed
>>with
>> return code 32] at
>>com.xensource.xenapi.Types.checkResponse(Types.java:2006)
>> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnec
>>tion.dispatch(XenServerConnectionPool.java:456)
>> at com.xensource.xenapi.SR.create(SR.java:801)
>> at
>>
>>
>>com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResou
>>rceBase.java:6129)
>> ... 17 more
>> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
>>Received:
>> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
>>Received:
>> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
>> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: {
>>Ans:
>> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
>> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch
>>Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
>> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
>> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags:
>>10, {
>> Answer } }
>> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing
>>class
>> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary
>> 2015-01-05 19:29:47,126 WARN
>> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
>> through host 1 due to Catch Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary
>> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store:
>>Can
>> not create storage pool through host 1 due to Catch Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary
>> com.cloud.utils.exception.CloudRuntimeException: Can not create storage
>> pool through host 1 due to Catch Exception
>> com.cloud.utils.exception.CloudRuntimeException, create StoragePool
>>failed
>> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
>> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
>> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
>> 192.168.161.110/mnt/data/VMS-TT/primary
>> at
>>
>>
>>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
>>oreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImp
>>l.java:384)
>> at
>>
>>
>>org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataSt
>>oreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.ja
>>va:407)
>> at
>>
>>
>>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:6
>>69)
>> at
>>
>>
>>com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:1
>>78)
>> at
>>
>>
>>org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.exec
>>ute(CreateStoragePoolCmd.java:163)
>> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
>> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
>> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
>> at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
>> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
>> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(D
>>efaultManagedContext.java:56)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWith
>>Context(DefaultManagedContext.java:103)
>> at
>>
>>
>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithC
>>ontext(DefaultManagedContext.java:53)
>> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
>> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>> at
>>
>>
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
>>ionFilterChain.java:290)
>> at
>>
>>
>>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
>>rChain.java:206)
>> at
>>
>>
>>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
>>.java:233)
>> at
>>
>>
>>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
>>.java:191)
>> at
>>
>>
>>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
>>127)
>> at
>>
>>
>>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
>>102)
>> at
>>
>>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
>> at
>>
>>
>>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
>>ava:109)
>> at
>>
>>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:29
>>8)
>> at
>>
>>
>>org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.ja
>>va:889)
>> at
>>
>>
>>org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.proces
>>s(Http11NioProtocol.java:721)
>> at
>>
>>
>>org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.ja
>>va:2274)
>> at
>>
>>
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
>>:1145)
>> at
>>
>>
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
>>a:615)
>> at java.lang.Thread.run(Thread.java:745)
>> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
>> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
>> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95
>>--
>> GET
>>
>>
>>command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&pod
>>Id=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158
>>-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%
>>2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyR
>>SPltz6ETNUkeE%3D&_=1420466385645
>> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
>> Primary Storage UPintenance mode
>>
>>
>> Same time Xenserver host showing following log..
>>
>>
>>
>> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
>> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ;
>>NFS
>> mount error [opterr=mount failed with return code 32]; ] Jan 5
>>19:29:47
>> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
>> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5
>>19:30:17
>> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
>> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
>> or NFS mount defined in /opt/cloud/bin/heartbeat!
>> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
>> or NFS mount defined in /opt/cloud/bin/heartbeat!
>> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
>> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
>>0.0.0.0:80|event.from
>> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
>>
>>
>> Tried to stop iptables on XS host
>> I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
>>
>>
>> Need help on this..
>>
>>
>>
>> Regards,
>> Tejas
>>
Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Tejas Gadaria <re...@gmail.com>.
Hi Suresh,
I was able to mount NFS on XS host through CLI mount.nfs4 successfully.
Also vhd-utils was present on Management server and XS host also. Still
facing same issue.
Regards,
Tejas
On Mon, Jan 5, 2015 at 11:16 PM, Suresh Sadhu <Su...@citrix.com>
wrote:
> Hope you have already copied vhd-util in to the host and also as Somesh
> suggested try manually mounting the storage directly from the host.
>
>
> Regards
> Sadhu
>
>
> -----Original Message-----
> From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
> Sent: 05 January 2015 20:16
> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
> error [opterr=mount failed with return code 32])
>
> XS is unable to create the SR failing with the following error:
> ---
> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
> return code 32]
> ---
>
> May be a problem with the NFS. Can you try manually creating a SR on XS
> using this NFS or may be simply try to mount the NFS on XS and see if that
> works.
>
> -----Original Message-----
> From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
> Sent: Monday, January 05, 2015 9:26 AM
> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
> error [opterr=mount failed with return code 32])
>
> Hi,
>
> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
>
>
> While creating Zone with NFS as primary & secondary storage
>
> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing request
> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-2:ctx-23cfb66b) Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6133)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:5074)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:475)
> at
>
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61)
> at
>
> com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
> at
>
> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
> return code 32] at com.xensource.xenapi.Types.checkResponse(Types.java:2006)
> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> at
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
> at com.xensource.xenapi.SR.create(SR.java:801)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6129)
> ... 17 more
> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: { Ans:
> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10, {
> Answer } }
> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing class
> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> 2015-01-05 19:29:47,126 WARN
> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
> through host 1 due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store: Can
> not create storage pool through host 1 due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> com.cloud.utils.exception.CloudRuntimeException: Can not create storage
> pool through host 1 due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> at
>
> org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImpl.java:384)
> at
>
> org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:407)
> at
>
> com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:669)
> at
>
> com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:178)
> at
>
> org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:163)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
> at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> at
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> at
>
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at
>
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> at
>
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95 --
> GET
>
> command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&podId=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyRSPltz6ETNUkeE%3D&_=1420466385645
> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
> Primary Storage UPintenance mode
>
>
> Same time Xenserver host showing following log..
>
>
>
> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ; NFS
> mount error [opterr=mount failed with return code 32]; ] Jan 5 19:29:47
> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5 19:30:17
> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> or NFS mount defined in /opt/cloud/bin/heartbeat!
> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> or NFS mount defined in /opt/cloud/bin/heartbeat!
> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
>
>
> Tried to stop iptables on XS host
> I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
>
>
> Need help on this..
>
>
>
> Regards,
> Tejas
>
Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Tejas Gadaria <re...@gmail.com>.
Hi Suresh,
I was able to mount NFS on XS host through CLI mount.nfs4 successfully.
Also vhd-utils was present on Management server and XS host also. Still
facing same issue.
Regards,
Tejas
On Mon, Jan 5, 2015 at 11:16 PM, Suresh Sadhu <Su...@citrix.com>
wrote:
> Hope you have already copied vhd-util in to the host and also as Somesh
> suggested try manually mounting the storage directly from the host.
>
>
> Regards
> Sadhu
>
>
> -----Original Message-----
> From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
> Sent: 05 January 2015 20:16
> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
> error [opterr=mount failed with return code 32])
>
> XS is unable to create the SR failing with the following error:
> ---
> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
> return code 32]
> ---
>
> May be a problem with the NFS. Can you try manually creating a SR on XS
> using this NFS or may be simply try to mount the NFS on XS and see if that
> works.
>
> -----Original Message-----
> From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
> Sent: Monday, January 05, 2015 9:26 AM
> To: users@cloudstack.apache.org; dev@cloudstack.apache.org
> Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
> error [opterr=mount failed with return code 32])
>
> Hi,
>
> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
>
>
> While creating Zone with NFS as primary & secondary storage
>
> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing request
> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-2:ctx-23cfb66b) Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6133)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:5074)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:475)
> at
>
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61)
> at
>
> com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
> at
>
> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
> return code 32] at com.xensource.xenapi.Types.checkResponse(Types.java:2006)
> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> at
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
> at com.xensource.xenapi.SR.create(SR.java:801)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6129)
> ... 17 more
> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: { Ans:
> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10, {
> Answer } }
> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing class
> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> 2015-01-05 19:29:47,126 WARN
> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
> through host 1 due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store: Can
> not create storage pool through host 1 due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> com.cloud.utils.exception.CloudRuntimeException: Can not create storage
> pool through host 1 due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> at
>
> org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImpl.java:384)
> at
>
> org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:407)
> at
>
> com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:669)
> at
>
> com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:178)
> at
>
> org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:163)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
> at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> at
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> at
>
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at
>
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> at
>
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95 --
> GET
>
> command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&podId=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyRSPltz6ETNUkeE%3D&_=1420466385645
> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
> Primary Storage UPintenance mode
>
>
> Same time Xenserver host showing following log..
>
>
>
> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ; NFS
> mount error [opterr=mount failed with return code 32]; ] Jan 5 19:29:47
> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5 19:30:17
> xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> or NFS mount defined in /opt/cloud/bin/heartbeat!
> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> or NFS mount defined in /opt/cloud/bin/heartbeat!
> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from
> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
>
>
> Tried to stop iptables on XS host
> I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
>
>
> Need help on this..
>
>
>
> Regards,
> Tejas
>
RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Suresh Sadhu <Su...@citrix.com>.
Hope you have already copied vhd-util in to the host and also as Somesh suggested try manually mounting the storage directly from the host.
Regards
Sadhu
-----Original Message-----
From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
Sent: 05 January 2015 20:16
To: users@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32])
XS is unable to create the SR failing with the following error:
---
Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32]
---
May be a problem with the NFS. Can you try manually creating a SR on XS using this NFS or may be simply try to mount the NFS on XS and see if that works.
-----Original Message-----
From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
Sent: Monday, January 05, 2015 9:26 AM
To: users@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32])
Hi,
I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
While creating Zone with NFS as primary & secondary storage
2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing request
2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-2:ctx-23cfb66b) Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6133)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:5074)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:475)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61)
at
com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32] at com.xensource.xenapi.Types.checkResponse(Types.java:2006)
at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
at com.xensource.xenapi.SR.create(SR.java:801)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6129)
... 17 more
2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: { Ans:
, MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10, { Answer } }
2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing class
com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
2015-01-05 19:29:47,126 WARN
[o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool through host 1 due to Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store: Can not create storage pool through host 1 due to Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
com.cloud.utils.exception.CloudRuntimeException: Can not create storage pool through host 1 due to Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
at
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImpl.java:384)
at
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:407)
at
com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:669)
at
com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:178)
at
org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:163)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95 -- GET
command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&podId=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyRSPltz6ETNUkeE%3D&_=1420466385645
2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no Primary Storage UPintenance mode
Same time Xenserver host showing following log..
xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ; NFS mount error [opterr=mount failed with return code 32]; ] Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or NFS mount defined in /opt/cloud/bin/heartbeat!
Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or NFS mount defined in /opt/cloud/bin/heartbeat!
Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
Tried to stop iptables on XS host
I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
Need help on this..
Regards,
Tejas
RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Suresh Sadhu <Su...@citrix.com>.
Hope you have already copied vhd-util in to the host and also as Somesh suggested try manually mounting the storage directly from the host.
Regards
Sadhu
-----Original Message-----
From: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
Sent: 05 January 2015 20:16
To: users@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32])
XS is unable to create the SR failing with the following error:
---
Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32]
---
May be a problem with the NFS. Can you try manually creating a SR on XS using this NFS or may be simply try to mount the NFS on XS and see if that works.
-----Original Message-----
From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
Sent: Monday, January 05, 2015 9:26 AM
To: users@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32])
Hi,
I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
While creating Zone with NFS as primary & secondary storage
2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing request
2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-2:ctx-23cfb66b) Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6133)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:5074)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:475)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61)
at
com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32] at com.xensource.xenapi.Types.checkResponse(Types.java:2006)
at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
at com.xensource.xenapi.SR.create(SR.java:801)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6129)
... 17 more
2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: { Ans:
, MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10, { Answer } }
2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing class
com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
2015-01-05 19:29:47,126 WARN
[o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool through host 1 due to Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store: Can not create storage pool through host 1 due to Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
com.cloud.utils.exception.CloudRuntimeException: Can not create storage pool through host 1 due to Catch Exception com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
at
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImpl.java:384)
at
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:407)
at
com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:669)
at
com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:178)
at
org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:163)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95 -- GET
command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&podId=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyRSPltz6ETNUkeE%3D&_=1420466385645
2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no Primary Storage UPintenance mode
Same time Xenserver host showing following log..
xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ; NFS mount error [opterr=mount failed with return code 32]; ] Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:17 xenserver-01 last message repeated 2 times Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or NFS mount defined in /opt/cloud/bin/heartbeat!
Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or NFS mount defined in /opt/cloud/bin/heartbeat!
Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET 0.0.0.0:80|event.from D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
Tried to stop iptables on XS host
I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
Need help on this..
Regards,
Tejas
RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Somesh Naidu <So...@citrix.com>.
XS is unable to create the SR failing with the following error:
---
Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
return code 32]
---
May be a problem with the NFS. Can you try manually creating a SR on XS using this NFS or may be simply try to mount the NFS on XS and see if that works.
-----Original Message-----
From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
Sent: Monday, January 05, 2015 9:26 AM
To: users@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32])
Hi,
I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
While creating Zone with NFS as primary & secondary storage
2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing request
2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-2:ctx-23cfb66b) Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6133)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:5074)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:475)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61)
at
com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
return code 32]
at com.xensource.xenapi.Types.checkResponse(Types.java:2006)
at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
at com.xensource.xenapi.SR.create(SR.java:801)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6129)
... 17 more
2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: { Ans:
, MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10, {
Answer } }
2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing class
com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
2015-01-05 19:29:47,126 WARN
[o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
through host 1 due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store: Can
not create storage pool through host 1 due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
com.cloud.utils.exception.CloudRuntimeException: Can not create storage
pool through host 1 due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
at
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImpl.java:384)
at
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:407)
at
com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:669)
at
com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:178)
at
org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:163)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95 --
GET
command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&podId=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyRSPltz6ETNUkeE%3D&_=1420466385645
2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no Primary
Storage UPintenance mode
Same time Xenserver host showing following log..
xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ; NFS
mount error [opterr=mount failed with return code 32]; ]
Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:17 xenserver-01 last message repeated 2 times
Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or
NFS mount defined in /opt/cloud/bin/heartbeat!
Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or
NFS mount defined in /opt/cloud/bin/heartbeat!
Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
Tried to stop iptables on XS host
I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
Need help on this..
Regards,
Tejas
Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Tejas Gadaria <re...@gmail.com>.
Hi Tejas.
I am able to mount on XS host through CLI "mount.nfs4" command.
I am using Ubuntu 12.04 Server as nfs server.
Regards,
Tejas
On Tue, Jan 6, 2015 at 12:54 PM, Tejas Sheth <ts...@gmail.com> wrote:
> Hi Tejas,
>
> Have you checked whether mannualy you are able to mount nfs share to xen
> server or not?
>
> If you are using nas box then check the allowed ip and its permissions.
>
> Thanks,
> Tejas
> On Jan 5, 2015 7:56 PM, "Tejas Gadaria" <re...@gmail.com> wrote:
>
> > Hi,
> >
> > I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
> >
> >
> > While creating Zone with NFS as primary & secondary storage
> >
> > 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
> > (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing request
> > 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
> > (DirectAgent-2:ctx-23cfb66b) Catch Exception
> > com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> failed
> > due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> > NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> > host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> > 192.168.161.110/mnt/data/VMS-TT/primary
> > com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
> > Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6133)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:5074)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:475)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
> > at
> >
> >
> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307)
> > at
> >
> >
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> > at
> >
> >
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> > at
> >
> >
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> > at
> >
> >
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> > at
> >
> >
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
> > return code 32]
> > at com.xensource.xenapi.Types.checkResponse(Types.java:2006)
> > at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
> > at com.xensource.xenapi.SR.create(SR.java:801)
> > at
> >
> >
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6129)
> > ... 17 more
> > 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> > (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
> Received:
> > 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> > (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response
> Received:
> > 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
> > (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: {
> Ans:
> > , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
> > [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch Exception
> > com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> failed
> > due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> > NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> > host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> > 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
> > 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
> > (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
> > Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
> {
> > Answer } }
> > 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
> > (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing class
> > com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
> > com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> failed
> > due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> > NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> > host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> > 192.168.161.110/mnt/data/VMS-TT/primary
> > 2015-01-05 19:29:47,126 WARN
> > [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
> > (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
> > through host 1 due to Catch Exception
> > com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> failed
> > due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> > NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> > host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> > 192.168.161.110/mnt/data/VMS-TT/primary
> > 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
> > (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store:
> Can
> > not create storage pool through host 1 due to Catch Exception
> > com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> failed
> > due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> > NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> > host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> > 192.168.161.110/mnt/data/VMS-TT/primary
> > com.cloud.utils.exception.CloudRuntimeException: Can not create storage
> > pool through host 1 due to Catch Exception
> > com.cloud.utils.exception.CloudRuntimeException, create StoragePool
> failed
> > due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> > NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> > host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> > 192.168.161.110/mnt/data/VMS-TT/primary
> > at
> >
> >
> org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImpl.java:384)
> > at
> >
> >
> org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:407)
> > at
> >
> >
> com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:669)
> > at
> >
> >
> com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:178)
> > at
> >
> >
> org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:163)
> > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
> > at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
> > at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
> > at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
> > at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
> > at
> >
> >
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> > at
> >
> >
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> > at
> >
> >
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
> > at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> > at
> >
> >
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> > at
> >
> >
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> > at
> >
> >
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> > at
> >
> >
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> > at
> >
> >
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> > at
> >
> >
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> > at
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> > at
> >
> >
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> > at
> >
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> > at
> >
> >
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> > at
> >
> >
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> > at
> >
> >
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > at
> >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > at java.lang.Thread.run(Thread.java:745)
> > 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
> > (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
> > 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
> > (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95 --
> > GET
> >
> >
> command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&podId=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyRSPltz6ETNUkeE%3D&_=1420466385645
> > 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> > (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no
> Primary
> > Storage UPintenance mode
> >
> >
> > Same time Xenserver host showing following log..
> >
> >
> >
> > xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
> > R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ;
> NFS
> > mount error [opterr=mount failed with return code 32]; ]
> > Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> > 0.0.0.0:80|event.from
> > D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
> > Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> > 0.0.0.0:80|event.from
> > D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
> > Jan 5 19:30:17 xenserver-01 last message repeated 2 times
> > Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> > 0.0.0.0:80|event.from
> > D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> > Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> > 0.0.0.0:80|event.from
> > D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> > Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> or
> > NFS mount defined in /opt/cloud/bin/heartbeat!
> > Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> > 0.0.0.0:80|event.from
> > D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> > Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> > 0.0.0.0:80|event.from
> > D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> > Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> > 0.0.0.0:80|event.from
> > D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> > Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> > 0.0.0.0:80|event.from
> > D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> > Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI
> or
> > NFS mount defined in /opt/cloud/bin/heartbeat!
> > Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> > 0.0.0.0:80|event.from
> > D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> > Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> > 0.0.0.0:80|event.from
> > D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
> >
> >
> > Tried to stop iptables on XS host
> > I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
> >
> >
> > Need help on this..
> >
> >
> >
> > Regards,
> > Tejas
> >
>
Re: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Tejas Sheth <ts...@gmail.com>.
Hi Tejas,
Have you checked whether mannualy you are able to mount nfs share to xen
server or not?
If you are using nas box then check the allowed ip and its permissions.
Thanks,
Tejas
On Jan 5, 2015 7:56 PM, "Tejas Gadaria" <re...@gmail.com> wrote:
> Hi,
>
> I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
>
>
> While creating Zone with NFS as primary & secondary storage
>
> 2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing request
> 2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
> (DirectAgent-2:ctx-23cfb66b) Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
> Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6133)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:5074)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:475)
> at
>
> com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61)
> at
>
> com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
> at
>
> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
>
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
> return code 32]
> at com.xensource.xenapi.Types.checkResponse(Types.java:2006)
> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> at
>
> com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
> at com.xensource.xenapi.SR.create(SR.java:801)
> at
>
> com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6129)
> ... 17 more
> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
> 2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
> 2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
> (DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: { Ans:
> , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
> 2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
> Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10, {
> Answer } }
> 2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing class
> com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> 2015-01-05 19:29:47,126 WARN
> [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
> through host 1 due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> 2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store: Can
> not create storage pool through host 1 due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> com.cloud.utils.exception.CloudRuntimeException: Can not create storage
> pool through host 1 due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
> NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
> host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
> 192.168.161.110/mnt/data/VMS-TT/primary
> at
>
> org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImpl.java:384)
> at
>
> org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:407)
> at
>
> com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:669)
> at
>
> com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:178)
> at
>
> org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:163)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
> at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
> at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
> at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
>
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> at
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> at
>
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at
>
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> at
>
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
> 2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
> (catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95 --
> GET
>
> command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&podId=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyRSPltz6ETNUkeE%3D&_=1420466385645
> 2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no Primary
> Storage UPintenance mode
>
>
> Same time Xenserver host showing following log..
>
>
>
> xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
> R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ; NFS
> mount error [opterr=mount failed with return code 32]; ]
> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> 0.0.0.0:80|event.from
> D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> 0.0.0.0:80|event.from
> D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:17 xenserver-01 last message repeated 2 times
> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> 0.0.0.0:80|event.from
> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> 0.0.0.0:80|event.from
> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or
> NFS mount defined in /opt/cloud/bin/heartbeat!
> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> 0.0.0.0:80|event.from
> D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> 0.0.0.0:80|event.from
> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> 0.0.0.0:80|event.from
> D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> 0.0.0.0:80|event.from
> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or
> NFS mount defined in /opt/cloud/bin/heartbeat!
> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> 0.0.0.0:80|event.from
> D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
> Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
> 0.0.0.0:80|event.from
> D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
>
>
> Tried to stop iptables on XS host
> I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
>
>
> Need help on this..
>
>
>
> Regards,
> Tejas
>
RE: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount
error [opterr=mount failed with return code 32])
Posted by Somesh Naidu <So...@citrix.com>.
XS is unable to create the SR failing with the following error:
---
Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
return code 32]
---
May be a problem with the NFS. Can you try manually creating a SR on XS using this NFS or may be simply try to mount the NFS on XS and see if that works.
-----Original Message-----
From: Tejas Gadaria [mailto:refond.gmrt@gmail.com]
Sent: Monday, January 05, 2015 9:26 AM
To: users@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: Unable to add primary storage (SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with return code 32])
Hi,
I am facing issue with xenserver 6.2 SP1, on ACS 4.3.1 and 4.4 both.
While creating Zone with NFS as primary & secondary storage
2015-01-05 19:29:45,715 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Executing request
2015-01-05 19:29:47,079 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-2:ctx-23cfb66b) Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
com.cloud.utils.exception.CloudRuntimeException: Unable to create NFS SR
Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary]
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6133)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:5074)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:475)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61)
at
com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:307)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: SR_BACKEND_FAILURE_73NFS mount error [opterr=mount failed with
return code 32]
at com.xensource.xenapi.Types.checkResponse(Types.java:2006)
at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
at com.xensource.xenapi.SR.create(SR.java:801)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNfsSR(CitrixResourceBase.java:6129)
... 17 more
2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
2015-01-05 19:29:47,080 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Response Received:
2015-01-05 19:29:47,080 DEBUG [c.c.a.t.Request]
(DirectAgent-2:ctx-23cfb66b) Seq 1-3875347479352311816: Processing: { Ans:
, MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":false,"details":"Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary","wait":0}}] }
2015-01-05 19:29:47,081 DEBUG [c.c.a.t.Request]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Seq 1-3875347479352311816:
Received: { Ans: , MgmtId: 158445874166998, via: 1, Ver: v1, Flags: 10, {
Answer } }
2015-01-05 19:29:47,081 DEBUG [c.c.a.m.AgentManagerImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Details from executing class
com.cloud.agent.api.CreateStoragePoolCommand: Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
2015-01-05 19:29:47,126 WARN
[o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Can not create storage pool
through host 1 due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
2015-01-05 19:29:47,126 DEBUG [c.c.s.StorageManagerImpl]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) Failed to add data store: Can
not create storage pool through host 1 due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
com.cloud.utils.exception.CloudRuntimeException: Can not create storage
pool through host 1 due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException, create StoragePool failed
due to com.cloud.utils.exception.CloudRuntimeException: Unable to create
NFS SR Pool[1|192.168.161.110:2049|/mnt/data/VMS-TT/primary] on
host:ba237c07-71cb-48a8-8a6d-6e6c3aff65b2 pool:
192.168.161.110/mnt/data/VMS-TT/primary
at
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.createStoragePool(CloudStackPrimaryDataStoreLifeCycleImpl.java:384)
at
org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.attachCluster(CloudStackPrimaryDataStoreLifeCycleImpl.java:407)
at
com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:669)
at
com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:178)
at
org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:163)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:682)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:511)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:330)
at com.cloud.api.ApiServlet.access$000(ApiServlet.java:54)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:118)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:115)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:77)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-01-05 19:29:47,129 INFO [c.c.a.ApiServer]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) null
2015-01-05 19:29:47,130 DEBUG [c.c.a.ApiServlet]
(catalina-exec-13:ctx-c00f6f19 ctx-37181993) ===END=== 192.168.160.95 --
GET
command=createStoragePool&zoneid=123cf76d-077a-4d92-a4e1-708aca573b1a&podId=5041c821-0d10-47df-a363-01f87cccfc00&clusterid=19698fe8-2921-48ea-9158-608cac69b5ac&name=Primary&scope=cluster&url=nfs%3A%2F%2F192.168.161.110%2Fmnt%2Fdata%2FVMS-TT%2Fprimary&response=json&sessionkey=qT%2FMwkWgbzKOyRSPltz6ETNUkeE%3D&_=1420466385645
2015-01-05 19:30:01,600 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-c4bb0888) Skip capacity scan due to there is no Primary
Storage UPintenance mode
Same time Xenserver host showing following log..
xenserver-01 xapi: [error|xenserver-01|926 INET 0.0.0.0:80|SR.create
R:adea9138f9b1|storage_access] Re-raising as SR_BACKEND_FAILURE_73 [ ; NFS
mount error [opterr=mount failed with return code 32]; ]
Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:c87d8d71bfdd|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:29:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:11d92819dc30|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:17 xenserver-01 last message repeated 2 times
Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or
NFS mount defined in /opt/cloud/bin/heartbeat!
Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:d575526e384a|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:30:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:6d4a45ec4259|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:17 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:43 xenserver-01 heartbeat: Problem with heartbeat, no iSCSI or
NFS mount defined in /opt/cloud/bin/heartbeat!
Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:0ecee91bf125|xapi] get_since_for_events: no in_memory_cache!
Jan 5 19:31:47 xenserver-01 xapi: [ warn|xenserver-01|866 INET
0.0.0.0:80|event.from
D:f473b74de5fd|xapi] get_since_for_events: no in_memory_cache!
Tried to stop iptables on XS host
I checked /opt/cloud/bin/heartbeat file, on XS host and file is empty.
Need help on this..
Regards,
Tejas