You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Devdeep Singh (JIRA)" <ji...@apache.org> on 2013/07/27 13:19:48 UTC
[jira] [Resolved] (CLOUDSTACK-3774) [XenServer]NPE while creating
template from snapshot on a upgraded setup
[ https://issues.apache.org/jira/browse/CLOUDSTACK-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devdeep Singh resolved CLOUDSTACK-3774.
---------------------------------------
Resolution: Cannot Reproduce
Tried it against an upgraded setup and we weren't able to reproduce the issue. The setup against which the issue was tried was upgraded from 2.2.15 to 4.2 (going through 3.0.6 in between). We did observe an issue in creating template from snapshot, but it was because vhd-util wasn't present in the updated 64 bit system vm. vhd-util not being present issue has already been fixed in another commit.
Looking at the logs it seems that the SSVM was in alert state from cloudstack. I also observe that the LocalNfsSecondaryStorageResource getting picked up. From the code it looks like this happens when we are not able to find any up and enabled SSVMs.
Please reopen the issue if you observe it again.
> [XenServer]NPE while creating template from snapshot on a upgraded setup
> ------------------------------------------------------------------------
>
> Key: CLOUDSTACK-3774
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3774
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Storage Controller, XenServer
> Affects Versions: 4.2.0
> Reporter: Sailaja Mada
> Assignee: Devdeep Singh
> Priority: Critical
> Fix For: 4.2.0
>
> Attachments: afterupgrademysqldumpusage1.dmp, apilog.log, beforeupgrade_mysqldump.dmp, management-server.log, snap.png, templatefromsnapshot.png
>
>
> Setup: Xen 6.1 hosts , Two zone ( Zone1 with local storage ,Zone 2 - NFS )
> Upgraded from 3.0.6 to 4.2
> Steps:
> 1) Create Account X and deploy VM on a NFS storage Zone 3
> 2) Create snapshot from ROOT volume of this instance
> 3) Tried to create template from this snapshot.
> Observations :
> 1. It failed with NPE.
> 2. UI results NPE as a failure message
> 2013-07-24 19:07:50,164 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots already exists on secondary storage, and is mounted at /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
> 2013-07-24 19:07:50,164 INFO [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots directory created/exists on Secondary Storage.
> 2013-07-24 19:07:50,165 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes already exists on secondary storage, and is mounted at /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
> 2013-07-24 19:07:50,166 INFO [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes directory created/exists on Secondary Storage.
> 2013-07-24 19:07:50,166 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: sudo mount
> 2013-07-24 19:07:50,224 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Execution is successful.
> 2013-07-24 19:07:50,224 DEBUG [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) mount point /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1 already exists
> 2013-07-24 19:07:50,309 DEBUG [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: null -p /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40 -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
> 2013-07-24 19:07:50,309 WARN [storage.resource.NfsSecondaryStorageResource] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Exception: null -p /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40 -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
> java.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
> at com.cloud.utils.script.Script.execute(Script.java:183)
> at com.cloud.utils.script.Script.execute(Script.java:161)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
> at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
> at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
> at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
> at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
> at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
> at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
> at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
> at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
> at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-24 19:07:50,348 DEBUG [cloud.template.TemplateManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Failed to create templatejava.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
> at com.cloud.utils.script.Script.execute(Script.java:183)
> at com.cloud.utils.script.Script.execute(Script.java:161)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
> at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
> at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
> at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
> at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
> at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
> at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
> at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
> at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
> at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
> at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
> at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
> at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-24 19:07:50,381 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Unexpected exception while executing org.apache.cloudstack.api.command.user.template.CreateTemplateCmd
> com.cloud.utils.exception.CloudRuntimeException: Failed to create templatejava.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
> at com.cloud.utils.script.Script.execute(Script.java:183)
> at com.cloud.utils.script.Script.execute(Script.java:161)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
> at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
> at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
> at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
> at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
> at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
> at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
> at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
> at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
> at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
> at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
> at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
> at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1369)
> at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-24 19:07:50,390 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Complete async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to create templatejava.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
> at com.cloud.utils.script.Script.execute(Script.java:183)
> at com.cloud.utils.script.Script.execute(Script.java:161)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
> at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
> at org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
> at org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
> at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
> at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
> at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
> at org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
> at org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
> at com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
> at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
> at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
> at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
> at org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-24 19:07:52,603 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===START=== 10.144.6.19 -- GET command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
> 2013-07-24 19:07:52,629 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-15:null) Async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ] completed
> 2013-07-24 19:07:52,640 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null) ===END=== 10.144.6.19 -- GET command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
> 2013-07-24 19:07:52,769 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-11:null) SeqA 10-8916: Processing Seq 10-8916: { Cmd , MgmtId: -1, via: 10, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":18,"_loadInfo":"{\n \"connections\": []\n}","wait":0}}] }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira