You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Animesh Chaturvedi (JIRA)" <ji...@apache.org> on 2014/11/02 17:48:34 UTC
[jira] [Updated] (CLOUDSTACK-6576) No Error Handling while
deploying GPU unsupported instances on a GPU enabled host (Ex: Cent OS
instance)
[ https://issues.apache.org/jira/browse/CLOUDSTACK-6576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Animesh Chaturvedi updated CLOUDSTACK-6576:
-------------------------------------------
Fix Version/s: 4.5.0
> No Error Handling while deploying GPU unsupported instances on a GPU enabled host (Ex: Cent OS instance)
> --------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-6576
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-6576
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Management Server, XenServer
> Affects Versions: 4.4.0
> Reporter: Sailaja Mada
> Assignee: Sanjay Tripathi
> Priority: Minor
> Fix For: 4.4.0, 4.5.0
>
> Attachments: logs1.zip
>
>
> Steps:
> 1. Install and configure Adv zone with GPU enabled XenServer 6.2.5
> 2. Try to deploy default Cent OS instance using GPU enabled offering( K2 - 200 profile)
> Observation:
> 1. Xenserver does not support vGPU with Linux instances.
> 2. Cloudstack throws deployment failure after throwing lot of exceptions by XenServer.
> Expected Results:
> CloudStack should parse the failure provided by XS and provide proper deployment failure as unsupported deployment.
> 2014-05-05 16:38:50,730 DEBUG [c.c.a.t.Request] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747) Seq 1-7036029992836286502: Sending { Cmd , MgmtId: 55355881446856, via: 1(cldstk-R720-66), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":9,"name":"i-3-9-VM","bootloader":"PyGrub","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":2147483648,"maxRam":2147483648,"arch":"x86_64","os":"CentOS 5.6 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"1439f05b06e154d2","params":{},"uuid":"00069b3f-abf0-4f0d-a12c-6ae9d1a3ea81","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e92dcd30-3775-4762-bb83-b6c9d901644b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e8ccf63a-577b-3378-9741-16ce76631992","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Sailaja/goletagpups1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Sailaja/goletagpups1/?ROLE=Primary&STOREUUID=e8ccf63a-577b-3378-9741-16ce76631992"}},"name":"ROOT-9","size":21474836480,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","volumeId":11,"vmName":"i-3-9-VM","accountId":3,"format":"VHD","id":11,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"91973ed0-97f6-47bf-94ea-25751da81ae9","ip":"10.1.1.220","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:03:4b:00:07","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://770","isolationUri":"vlan://770","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K200"}},"hostIp":"10.102.192.66","executeInSequence":false,"wait":0}}] }
> 2014-05-05 16:38:50,732 DEBUG [c.c.a.t.Request] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747) Seq 1-7036029992836286502: Executing: { Cmd , MgmtId: 55355881446856, via: 1(cldstk-R720-66), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":9,"name":"i-3-9-VM","bootloader":"PyGrub","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":2147483648,"maxRam":2147483648,"arch":"x86_64","os":"CentOS 5.6 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"1439f05b06e154d2","params":{},"uuid":"00069b3f-abf0-4f0d-a12c-6ae9d1a3ea81","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e92dcd30-3775-4762-bb83-b6c9d901644b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e8ccf63a-577b-3378-9741-16ce76631992","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Sailaja/goletagpups1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Sailaja/goletagpups1/?ROLE=Primary&STOREUUID=e8ccf63a-577b-3378-9741-16ce76631992"}},"name":"ROOT-9","size":21474836480,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","volumeId":11,"vmName":"i-3-9-VM","accountId":3,"format":"VHD","id":11,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"91973ed0-97f6-47bf-94ea-25751da81ae9","ip":"10.1.1.220","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:03:4b:00:07","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://770","isolationUri":"vlan://770","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K200"}},"hostIp":"10.102.192.66","executeInSequence":false,"wait":0}}] }
> 2014-05-05 16:38:50,732 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-296:ctx-251a7963) Seq 1-7036029992836286502: Executing request
> 2014-05-05 16:38:50,743 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) 1. The VM i-3-9-VM is in Starting state.
> 2014-05-05 16:38:50,780 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Created VM 2b2d13d8-42ea-301d-6b65-0ad093195826 for i-3-9-VM
> 2014-05-05 16:38:50,794 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Creating VGPU for of VGPU type: GRID K200 in GPU group Group of NVIDIA Corporation GK104GL [GRID K2] GPUs for VM i-3-9-VM
> 2014-05-05 16:38:50,794 DEBUG [c.c.h.x.r.XenServer620SP1Resource] (DirectAgent-296:ctx-251a7963) Creating VGPU of VGPU type [ GRID K200 ] in gpu groupGroup of NVIDIA Corporation GK104GL [GRID K2] GPUs for VM i-3-9-VM
> 2014-05-05 16:38:50,818 DEBUG [c.c.h.x.r.XenServer620SP1Resource] (DirectAgent-296:ctx-251a7963) Created VGPU of VGPU type [ GRID K200 ] for VM i-3-9-VM
> 2014-05-05 16:38:50,954 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) VBD 74e87e65-0066-f400-eb20-e322c0c64ebe created for com.cloud.agent.api.to.DiskTO@61ba9994
> 2014-05-05 16:38:50,975 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) VBD d4e27c31-8ef4-49ca-661c-712f83568a03 created for com.cloud.agent.api.to.DiskTO@21b8f5ac
> 2014-05-05 16:38:50,975 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Creating VIF for i-3-9-VM on nic [Nic:Guest-10.1.1.220-vlan://770]
> 2014-05-05 16:38:51,000 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Found a network called VLAN-0a144be7-8521-934e-ff53-c62c59e75d69-770 on host=10.102.192.66; Network=fa62257b-e53e-1e82-69f0-963e63f853e0; pif=4fb4b5fd-c11c-5827-8198-8861506bda56
> 2014-05-05 16:38:51,011 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Created a vif c6b0df48-0a58-9640-5d23-302cf3c3d7b4 on 0
> 2014-05-05 16:38:52,033 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Task failed! Task record: uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
> nameLabel: Async.VM.start_on
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Mon May 05 15:28:17 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@be7936ed
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> 2014-05-05 16:38:52,041 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Unable to start VM(i-3-9-VM) on host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
> nameLabel: Async.VM.start_on
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Mon May 05 15:28:17 IST 2014
> finished: Mon May 05 15:28:17 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@be7936ed
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> Task failed! Task record: uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
> nameLabel: Async.VM.start_on
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Mon May 05 15:28:17 IST 2014
> finished: Mon May 05 15:28:17 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@be7936ed
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:3346)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:3458)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1812)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
> at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
> at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
> at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2014-05-05 16:38:52,046 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) unable to destroy task(com.xensource.xenapi.Task@d080c3e6) on host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
> 2014-05-05 16:38:52,046 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Catch Exception: class com.cloud.utils.exception.CloudRuntimeException due to com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(i-3-9-VM) on host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
> nameLabel: Async.VM.start_on
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Mon May 05 15:28:17 IST 2014
> finished: Mon May 05 15:28:17 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@be7936ed
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(i-3-9-VM) on host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
> nameLabel: Async.VM.start_on
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Mon May 05 15:28:17 IST 2014
> finished: Mon May 05 15:28:17 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@be7936ed
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:3477)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1812)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
> at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
> at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
> at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2014-05-05 16:38:52,048 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Unable to start i-3-9-VM due to
> com.cloud.utils.exception.CloudRuntimeException: Unable to start VM(i-3-9-VM) on host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record: uuid: fe8bd262-4b47-9a13-ac8e-1c2cafbea4fb
> nameLabel: Async.VM.start_on
> nameDescription:
> allowedOperations: []
> currentOperations: {}
> created: Mon May 05 15:28:17 IST 2014
> finished: Mon May 05 15:28:17 IST 2014
> status: failure
> residentOn: com.xensource.xenapi.Host@be7936ed
> progress: 1.0
> type: <none/>
> result:
> errorInfo: [FEATURE_REQUIRES_HVM, vGPU- and GPU-passthrough needs HVM]
> otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f
> subtasks: []
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:3477)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1812)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
> at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
> at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
> at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2014-05-05 16:38:52,060 DEBUG [c.c.a.ApiServlet] (catalina-exec-13:ctx-1d7ac3bf) ===START=== 10.144.6.6 -- GET command=queryAsyncJobResult&jobId=48d1d975-c105-4823-99e1-6fc5f51f180e&response=json&sessionkey=TDXLIOmV8uKaEsb9Mr5o96exjw0%3D&_=1399288191562
> 2014-05-05 16:38:52,083 DEBUG [c.c.a.ApiServlet] (catalina-exec-13:ctx-1d7ac3bf ctx-6f979a7b) ===END=== 10.144.6.6 -- GET command=queryAsyncJobResult&jobId=48d1d975-c105-4823-99e1-6fc5f51f180e&response=json&sessionkey=TDXLIOmV8uKaEsb9Mr5o96exjw0%3D&_=1399288191562
> 2014-05-05 16:38:52,105 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Unable to clean up VBD due to
> You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
> at com.xensource.xenapi.Types.checkResponse(Types.java:693)
> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
> at com.xensource.xenapi.VBD.unplug(VBD.java:1059)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1542)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1892)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
> at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
> at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
> at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2014-05-05 16:38:52,112 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Unable to clean up VBD due to
> You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
> at com.xensource.xenapi.Types.checkResponse(Types.java:693)
> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
> at com.xensource.xenapi.VBD.unplug(VBD.java:1059)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1542)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1892)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
> at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
> at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
> at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2014-05-05 16:38:52,119 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) Unable to cleanup VIF
> You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
> at com.xensource.xenapi.Types.checkResponse(Types.java:693)
> at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
> at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
> at com.xensource.xenapi.VIF.unplug(VIF.java:878)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1550)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1892)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:500)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:60)
> at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:93)
> at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:64)
> at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2014-05-05 16:38:52,137 INFO [c.c.h.x.r.XenServer56Resource] (DirectAgent-296:ctx-251a7963) Catch com.xensource.xenapi.Types$VifInUse: failed to destory VLAN eth0 on host 76520b97-5ce8-4c71-9572-b0834b62dc35 due to Network has active VIFs
> 2014-05-05 16:38:52,137 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-296:ctx-251a7963) The VM is in stopped state, detected problem during startup : i-3-9-VM
> 2014-05-05 16:38:52,137 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-296:ctx-251a7963) Seq 1-7036029992836286502: Response Received:
> 2014-05-05 16:38:52,139 DEBUG [c.c.a.t.Request] (DirectAgent-296:ctx-251a7963) Seq 1-7036029992836286502: Processing: { Ans: , MgmtId: 55355881446856, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":9,"name":"i-3-9-VM","bootloader":"PyGrub","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":2147483648,"maxRam":2147483648,"arch":"x86_64","os":"CentOS 5.6 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"1439f05b06e154d2","params":{},"uuid":"00069b3f-abf0-4f0d-a12c-6ae9d1a3ea81","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"e92dcd30-3775-4762-bb83-b6c9d901644b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e8ccf63a-577b-3378-9741-16ce76631992","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Sailaja/goletagpups1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Sailaja/goletagpups1/?ROLE=Primary&STOREUUID=e8ccf63a-577b-3378-9741-16ce76631992"}},"name":"ROOT-9","size":21474836480,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","volumeId":11,"vmName":"i-3-9-VM","accountId":3,"format":"VHD","id":11,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8d273d72-877c-4776-b4b5-9e52072ea45f","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"91973ed0-97f6-47bf-94ea-25751da81ae9","ip":"10.1.1.220","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:03:4b:00:07","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://770","isolationUri":"vlan://770","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID K200","groupDetails":{"Group of NVIDIA Corporation GK104GL [GRID K2] GPUs":{"GRID K200":{"modelName":"GRID K200","maxHeads":2,"videoRam":268435456,"maxResolutionX":1920,"maxResolutionY":1200,"maxVgpuPerGpu":8,"remainingCapacity":16,"maxCapacity":16},"passthrough":{"modelName":"passthrough","maxHeads":0,"videoRam":0,"maxResolutionX":0,"maxResolutionY":0,"maxVgpuPerGpu":1,"remainingCapacity":2,"maxCapacity":2},"GRID K260Q":{"modelName":"GRID K260Q","maxHeads":4,"videoRam":2013265920,"maxResolutionX":2560,"maxResolutionY":1600,"maxVgpuPerGpu":2,"remainingCapacity":4,"maxCapacity":4},"GRID K240Q":{"modelName":"GRID K240Q","maxHeads":2,"videoRam":1006632960,"maxResolutionX":2560,"maxResolutionY":1600,"maxVgpuPerGpu":4,"remainingCapacity":8,"maxCapacity":8}}}}},"_iqnToPath":{},"result":false,"details":"Unable to start i-3-9-VM due to ","wait":0}}] }
> 2014-05-05 16:38:52,139 DEBUG [c.c.a.t.Request] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747) Seq 1-7036029992836286502: Received: { Ans: , MgmtId: 55355881446856, via: 1, Ver: v1, Flags: 10, { StartAnswer } }
> 2014-05-05 16:38:52,142 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747) Unable to start VM on Host[-1-Routing] due to Unable to start i-3-9-VM due to
> 2014-05-05 16:38:52,145 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747) Cleaning up resources for the vm VM[User|i-3-9-VM] in Starting state
> 2014-05-05 16:38:52,148 DEBUG [c.c.a.t.Request] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747) Seq 1-7036029992836286503: Sending { Cmd , MgmtId: 55355881446856, via: 1(cldstk-R720-66), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-3-9-VM","wait":0}}] }
> 2014-05-05 16:38:52,148 DEBUG [c.c.a.t.Request] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747) Seq 1-7036029992836286503: Executing: { Cmd , MgmtId: 55355881446856, via: 1(cldstk-R720-66), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-3-9-VM","wait":0}}] }
> 2014-05-05 16:38:52,224 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-3-9-VM]Scope=interface com.cloud.dc.DataCenter; id=1
> 2014-05-05 16:38:52,225 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-18:job-58/job-59 ctx-4a14e747) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-3-9-VM]Scope=interface com.cloud.dc.DataCenter; id=1
> 2014-05-05 16:38:52,226 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-18:job-58/job-59) Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 9, job origin: 58
> 2014-05-05 16:38:52,226 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-18:job-58/job-59) Unable to complete AsyncJobVO {id:59, userId: 3, accountId: 3, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAADAAAAAAAAAAMAAAAAAAAACXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAAFxAH4ACnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 55355881446856, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon May 05 16:38:46 IST 2014}, job origin:58
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-3-9-VM]Scope=interface com.cloud.dc.DataCenter; id=1
> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:943)
> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5167)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5312)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:495)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:452)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2014-05-05 16:38:52,228 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-18:job-58/job-59) Complete async job-59, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlAgAAeHIAE2phdmEubGFuZy5FeGNlcHRpb27Q_R8-GjscxAIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TAAUc3VwcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBxAH4AB3QAT0pvYiBmYWlsZWQgZHVlIHRvIGV4Y2VwdGlvbiBVbmFibGUgdG8gY3JlYXRlIGEgZGVwbG95bWVudCBmb3IgVk1bVXNlcnxpLTMtOS1WTV11cgAeW0xqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnQ7AkYqPDz9IjkCAAB4cAAAAA1zcgAbamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50YQnFmiY23YUCAARJAApsaW5lTnVtYmVyTAAOZGVjbGFyaW5nQ2xhc3NxAH4ABEwACGZpbGVOYW1lcQB-AARMAAptZXRob2ROYW1lcQB-AAR4cAAAAHJ0ACBjb20uY2xvdWQudm0uVm1Xb3JrSm9iRGlzcGF0Y2hlcnQAGFZtV29ya0pvYkRpc3BhdGNoZXIuamF2YXQABnJ1bkpvYnNxAH4ACwAAAe90AD9vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZnJhbWV3b3JrLmpvYnMuaW1wbC5Bc3luY0pvYk1hbmFnZXJJbXBsJDV0ABhBc3luY0pvYk1hbmFnZXJJbXBsLmphdmF0AAxydW5JbkNvbnRleHRzcQB-AAsAAAAxdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm5hYmxlLmphdmF0AANydW5zcQB-AAsAAAA4dABCb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5pbXBsLkRlZmF1bHRNYW5hZ2VkQ29udGV4dCQxdAAaRGVmYXVsdE1hbmFnZWRDb250ZXh0LmphdmF0AARjYWxsc3EAfgALAAAAZ3QAQG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHRxAH4AGnQAD2NhbGxXaXRoQ29udGV4dHNxAH4ACwAAADVxAH4AHXEAfgAadAAOcnVuV2l0aENvbnRleHRzcQB-AAsAAAAudAA8b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlcQB-ABZxAH4AF3NxAH4ACwAAAcRxAH4AEXEAfgAScQB-ABdzcQB-AAsAAAHXdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuRXhlY3V0b3JzJFJ1bm5hYmxlQWRhcHRlcnQADkV4ZWN1dG9ycy5qYXZhcQB-ABtzcQB-AAsAAAEGdAAfamF2YS51dGlsLmNvbmN1cnJlbnQuRnV0dXJlVGFza3QAD0Z1dHVyZVRhc2suamF2YXEAfgAXc3EAfgALAAAEeXQAJ2phdmEudXRpbC5jb25jdXJyZW50LlRocmVhZFBvb2xFeGVjdXRvcnQAF1RocmVhZFBvb2xFeGVjdXRvci5qYXZhdAAJcnVuV29ya2Vyc3EAfgALAAACZ3QALmphdmEudXRpbC5jb25jdXJyZW50LlRocmVhZFBvb2xFeGVjdXRvciRXb3JrZXJxAH4ALHEAfgAXc3EAfgALAAAC6HQAEGphdmEubGFuZy5UaHJlYWR0AAtUaHJlYWQuamF2YXEAfgAXc3IAJmphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVMaXN0_A8lMbXsjhACAAFMAARsaXN0cQB-AAZ4cgAsamF2YS51dGlsLkNvbGxlY3Rpb25zJFVubW9kaWZpYWJsZUNvbGxlY3Rpb24ZQgCAy173HgIAAUwAAWN0ABZMamF2YS91dGlsL0NvbGxlY3Rpb247eHBzcgATamF2YS51dGlsLkFycmF5TGlzdHiB0h2Zx2GdAwABSQAEc2l6ZXhwAAAAAHcEAAAAAHhxAH4AOHg
> CCP should
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)