You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Devdeep Singh (JIRA)" <ji...@apache.org> on 2013/12/25 10:51:50 UTC
[jira] [Assigned] (CLOUDSTACK-5639) [Hyper-v] cold storage
migration does not work in hyperv
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devdeep Singh reassigned CLOUDSTACK-5639:
-----------------------------------------
Assignee: Devdeep Singh
> [Hyper-v] cold storage migration does not work in hyperv
> --------------------------------------------------------
>
> Key: CLOUDSTACK-5639
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5639
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Hypervisor Controller, Management Server
> Affects Versions: 4.3.0
> Environment: Latest build from 4.3 branch with commit:4b2b6544255ee7fe12b4b4732cacd08e4ee6a1b7
> Storage: SMB for both primary and secondary
> Hypervisor: Hyper-v
> Reporter: Sanjeev N
> Assignee: Devdeep Singh
> Priority: Critical
> Labels: hyper-V,, hyper-v, hyperv
> Fix For: 4.3.0
>
> Attachments: management-server.rar
>
>
> [Hyper-v] cold storage does not work in hyperv
> Steps to Reproduce:
> ================
> 1.Bring up CS in advanced zone with 2 hosts in a hyper-v cluster using CIFS for both primary and secondary storage
> 2.Deploy one or two guest vms using default cent os template.
> 3.Add one more primary storage (say primary2) to the cluster
> 4.Stop one of the vms deployed at step2
> 5.Migrate the storage from source primary to new primary storage primary2
> Expected Result:
> =============
> Storage migration should succeed when the vm is in stopped state
> Actual Result:
> ===========
> Storage migration failed with the error "Failed to migration: com.cloud.exception.StorageUnavailableException: Resource [StoragePool:7] is unreachable"
> Observations:
> ============
> Log snippet from the MS log:
> 2013-12-24 19:11:55,475 DEBUG [c.c.a.ApiServlet] (catalina-exec-11:ctx-717ba369) ===START=== 10.146.0.134 -- GET command=migrateVirtualMachine&storageid=fe97ca45-5cba-329a-ae79-fa02649421c1&virtualmachineid=3a1b4f23-7a43-4806-8fd5-43a9c1130872&response=json&sessionkey=uHhzduF1EppEHm%2Bibsj2%2Bqf%2Fzto%3D&_=1387892514178
> 2013-12-24 19:11:55,661 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (catalina-exec-11:ctx-717ba369 ctx-d805adb4) submit async job-33, details: AsyncJobVO {id:33, userId: 2, accountId: 2, instanceType: None, instanceId: null, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdInfo: {"response":"json","sessionkey":"uHhzduF1EppEHm+ibsj2+qf/zto\u003d","virtualmachineid":"3a1b4f23-7a43-4806-8fd5-43a9c1130872","cmdEventType":"VM.MIGRATE","ctxUserId":"2","storageid":"fe97ca45-5cba-329a-ae79-fa02649421c1","httpmethod":"GET","_":"1387892514178","ctxAccountId":"2","ctxStartEventId":"99"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 132129494109518, completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2013-12-24 19:11:55,663 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-29:ctx-893d8661) Add job-33 into job monitoring
> 2013-12-24 19:11:55,663 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-29:ctx-893d8661) Executing AsyncJobVO {id:33, userId: 2, accountId: 2, instanceType: None, instanceId: null, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdInfo: {"response":"json","sessionkey":"uHhzduF1EppEHm+ibsj2+qf/zto\u003d","virtualmachineid":"3a1b4f23-7a43-4806-8fd5-43a9c1130872","cmdEventType":"VM.MIGRATE","ctxUserId":"2","storageid":"fe97ca45-5cba-329a-ae79-fa02649421c1","httpmethod":"GET","_":"1387892514178","ctxAccountId":"2","ctxStartEventId":"99"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 132129494109518, completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2013-12-24 19:11:55,663 DEBUG [c.c.a.ApiServlet] (catalina-exec-11:ctx-717ba369 ctx-d805adb4) ===END=== 10.146.0.134 -- GET command=migrateVirtualMachine&storageid=fe97ca45-5cba-329a-ae79-fa02649421c1&virtualmachineid=3a1b4f23-7a43-4806-8fd5-43a9c1130872&response=json&sessionkey=uHhzduF1EppEHm%2Bibsj2%2Bqf%2Fzto%3D&_=1387892514178
> 2013-12-24 19:11:55,794 DEBUG [c.c.c.CapacityManagerImpl] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) VM state transitted from :Stopped to Migrating with event: StorageMigrationRequestedvm's original host id: 4 new host id: null host id before state transition: null
> 2013-12-24 19:11:56,026 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) copyAsync inspecting src type VOLUME copyAsync inspecting dest type VOLUME
> 2013-12-24 19:11:56,199 DEBUG [c.c.a.t.Request] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Seq 1-1652228440: Sending { Cmd , MgmtId: 132129494109518, via: 1(10.147.40.14), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"34a817b2-e24f-4fb3-82dc-918505fcf8c8","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9e132ff7-f9ef-3f17-bbc2-c44c323d96e7","id":1,"poolType":"NetworkFilesystem","host":"Host13","path":"/hyperv/primary?user=sanjeev&password=Password_123&domain=BLR","port":445,"url":"NetworkFilesystem://Host13//hyperv/primary?user=sanjeev&password=Password_123&domain=BLR/?ROLE=Primary&STOREUUID=9e132ff7-f9ef-3f17-bbc2-c44c323d96e7"}},"name":"ROOT-3","size":5368709120,"volumeId":3,"vmName":"i-2-3-VM","accountId":2,"id":3,"deviceId":0,"hypervisorType":"Hyperv"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"34a817b2-e24f-4fb3-82dc-918505fcf8c8","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR","_role":"Image"}},"name":"ROOT-3","size":5368709120,"path":"volumes/2/3","volumeId":3,"vmName":"i-2-3-VM","accountId":2,"id":3,"deviceId":0,"hypervisorType":"Hyperv"}},"executeInSequence":false,"options":{},"wait":10800}}] }
> 2013-12-24 19:11:56,199 DEBUG [c.c.a.t.Request] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Seq 1-1652228440: Executing: { Cmd , MgmtId: 132129494109518, via: 1(10.147.40.14), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"34a817b2-e24f-4fb3-82dc-918505fcf8c8","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9e132ff7-f9ef-3f17-bbc2-c44c323d96e7","id":1,"poolType":"NetworkFilesystem","host":"Host13","path":"/hyperv/primary?user=sanjeev&password=Password_123&domain=BLR","port":445,"url":"NetworkFilesystem://Host13//hyperv/primary?user=sanjeev&password=Password_123&domain=BLR/?ROLE=Primary&STOREUUID=9e132ff7-f9ef-3f17-bbc2-c44c323d96e7"}},"name":"ROOT-3","size":5368709120,"volumeId":3,"vmName":"i-2-3-VM","accountId":2,"id":3,"deviceId":0,"hypervisorType":"Hyperv"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"34a817b2-e24f-4fb3-82dc-918505fcf8c8","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR","_role":"Image"}},"name":"ROOT-3","size":5368709120,"path":"volumes/2/3","volumeId":3,"vmName":"i-2-3-VM","accountId":2,"id":3,"deviceId":0,"hypervisorType":"Hyperv"}},"executeInSequence":false,"options":{},"wait":10800}}] }
> 2013-12-24 19:11:56,208 DEBUG [c.c.a.t.Request] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Seq 1-1652228440: Received: { Ans: , MgmtId: 132129494109518, via: 1, Ver: v1, Flags: 10, { CopyCmdAnswer } }
> 2013-12-24 19:11:56,208 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) copy to image store failed: org.apache.cloudstack.storage.command.CopyCommand failed on exception, Value cannot be null.^M
> Parameter name: VolumeObjectTO missing primary dataStore in spec {^M
> "uuid": "34a817b2-e24f-4fb3-82dc-918505fcf8c8",^M
> "volumeType": "ROOT",^M
> "dataStore": {^M
> "com.cloud.agent.api.to.NfsTO": {^M
> "_url": "cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR",^M
> "_role": "Image"^M
> }^M
> },^M
> "name": "ROOT-3",^M
> "size": 5368709120,^M
> "path": "volumes/2/3",^M
> "volumeId": 3,^M
> "vmName": "i-2-3-VM",^M
> "accountId": 2,^M
> "id": 3,^M
> "deviceId": 0,^M
> "hypervisorType": "Hyperv"^M
> }2013-12-24 19:11:56,394 DEBUG [c.c.a.t.Request] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Seq 3-177144205: Sending { Cmd , MgmtId: 132129494109518, via: 3(s-2-VM), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"34a817b2-e24f-4fb3-82dc-918505fcf8c8","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR","_role":"Image"}},"name":"ROOT-3","size":5368709120,"volumeId":3,"vmName":"i-2-3-VM","accountId":2,"id":3,"deviceId":0,"hypervisorType":"Hyperv"}},"wait":0}}] }
> 2013-12-24 19:11:56,438 DEBUG [c.c.a.t.Request] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Seq 3-177144205: Received: { Ans: , MgmtId: 132129494109518, via: 3, Ver: v1, Flags: 10, { Answer } }
> 2013-12-24 19:11:56,439 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Volume 3 is not found on image store 1, so no need to delete
> 2013-12-24 19:11:56,505 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@10d466d5), no need to delete from object in store ref table
> 2013-12-24 19:11:56,735 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Marking volume that was never created as destroyed: Vol[9|vm=3|ROOT]
> 2013-12-24 19:11:56,788 ERROR [o.a.c.e.o.VolumeOrchestrator] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) migrate volume failed:org.apache.cloudstack.storage.command.CopyCommand failed on exception, Value cannot be null.
> Parameter name: VolumeObjectTO missing primary dataStore in spec {^M
> "uuid": "34a817b2-e24f-4fb3-82dc-918505fcf8c8",^M
> "volumeType": "ROOT",^M
> "dataStore": {^M
> "com.cloud.agent.api.to.NfsTO": {^M
> "_url": "cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR",^M
> "_role": "Image"^M
> }^M
> },^M
> "name": "ROOT-3",^M
> "size": 5368709120,^M
> "path": "volumes/2/3",^M
> "volumeId": 3,^M
> "vmName": "i-2-3-VM",^M
> "accountId": 2,^M
> "id": 3,^M
> "deviceId": 0,^M
> "hypervisorType": "Hyperv"^M
> }
> 2013-12-24 19:11:56,788 DEBUG [c.c.v.VirtualMachineManagerImpl] (Job-Executor-29:ctx-893d8661 ctx-d805adb4) Failed to migration: com.cloud.exception.StorageUnavailableException: Resource [StoragePool:7] is unreachable: migrate volume failed: org.apache.cloudstack.storage.command.CopyCommand failed on exception, Value cannot be null.^M
> Agent log:
> ========
> 2013-12-24 11:25:55,436 [6] ERROR HypervResource.HypervResourceController [d72f6849-8022-4183-87f1-3202083ed907] - com.cloud.agent.api.GetStorageStatsCommand failed on exceptionIllegal characters in path.^M
> System.ArgumentException: Illegal characters in path.^M
> at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)^M
> at System.Security.Permissions.FileIOPermission.CheckIllegalCharacters(String[] str)^M
> at System.Security.Permissions.FileIOPermission.AddPathList(FileIOPermissionAccess access, AccessControlActions control, String[] pathListOrig, Boolean checkForDuplicates, Boolean needFullPath, Boolean copyPathList)^M
> at System.Security.Permissions.FileIOPermission..ctor(FileIOPermissionAccess access, String[] pathList, Boolean checkForDuplicates, Boolean needFullPath)^M
> at System.IO.Path.GetFullPath(String path)^M
> at HypervResource.HypervResourceController.GetCapacityForLocalPath(String localStoragePath, Int64& capacityBytes, Int64& availableBytes)^M
> at HypervResource.HypervResourceController.GetStorageStatsCommand(Object cmd)^M
> 2013-12-24 11:25:55,436 [6] INFO HypervResource.HypervResourceController [d72f6849-8022-4183-87f1-3202083ed907] - {^M
> "com.cloud.agent.api.GetStorageStatsAnswer": {^M
> "result": false,^M
> "details": "com.cloud.agent.api.GetStorageStatsCommand failed on exceptionIllegal characters in path.",^M
> "capacity": 0,^M
> "used": 0,^M
> "contextMap": {}^M
> }^M
> }^M
> 2013-12-24 11:26:20,795 [35] ERROR HypervResource.VolumeObjectTO [1d5b7262-afc4-4b65-ab5a-a5b189817c5e] - VolumeObjectTO missing primary dataStore in spec {^M
> "uuid": "34a817b2-e24f-4fb3-82dc-918505fcf8c8",^M
> "volumeType": "ROOT",^M
> "dataStore": {^M
> "com.cloud.agent.api.to.NfsTO": {^M
> "_url": "cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR",^M
> "_role": "Image"^M
> }^M
> },^M
> "name": "ROOT-3",^M
> "size": 5368709120,^M
> "path": "volumes/2/3",^M
> "volumeId": 3,^M
> "vmName": "i-2-3-VM",^M
> "accountId": 2,^M
> "id": 3,^M
> "deviceId": 0,^M
> "hypervisorType": "Hyperv"^M
> }^M
> 2013-12-24 11:26:20,795 [35] ERROR HypervResource.HypervResourceController [1d5b7262-afc4-4b65-ab5a-a5b189817c5e] - org.apache.cloudstack.storage.command.CopyCommand failed on exception, Value cannot be null.^M
> Parameter name: VolumeObjectTO missing primary dataStore in spec {^M
> "uuid": "34a817b2-e24f-4fb3-82dc-918505fcf8c8",^M
> "volumeType": "ROOT",^M
> "dataStore": {^M
> "com.cloud.agent.api.to.NfsTO": {^M
> "_url": "cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR",^M
> "_role": "Image"^M
> }^M
> },^M
> "name": "ROOT-3",^M
> "size": 5368709120,^M
> "path": "volumes/2/3",^M
> "volumeId": 3,^M
> "vmName": "i-2-3-VM",^M
> "accountId": 2,^M
> "id": 3,^M
> "deviceId": 0,^M
> "hypervisorType": "Hyperv"^M
> }^M
> System.ArgumentNullException: Value cannot be null.^M
> Parameter name: VolumeObjectTO missing primary dataStore in spec {^M
> "uuid": "34a817b2-e24f-4fb3-82dc-918505fcf8c8",^M
> "volumeType": "ROOT",^M
> "dataStore": {^M
> "com.cloud.agent.api.to.NfsTO": {^M
> "_url": "cifs://10.147.40.13/hyperv/secondary?user=sanjeev&password=Password_123&domain=BLR",^M
> "_role": "Image"^M
> }^M
> },^M
> "name": "ROOT-3",^M
> "size": 5368709120,^M
> "path": "volumes/2/3",^M
> "volumeId": 3,^M
> "vmName": "i-2-3-VM",^M
> "accountId": 2,^M
> "id": 3,^M
> "deviceId": 0,^M
> "hypervisorType": "Hyperv"^M
> }^M
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)