You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Rayees Namathponnan (JIRA)" <ji...@apache.org> on 2013/07/17 22:41:47 UTC

[jira] [Comment Edited] (CLOUDSTACK-3538) [Automation]Router and SSVM not coming up after restart

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13711583#comment-13711583 ] 

Rayees Namathponnan edited comment on CLOUDSTACK-3538 at 7/17/13 8:40 PM:
--------------------------------------------------------------------------

Yes, i didnt find the issue with latest build 

But as per Edison, this is not a complete fix, we need to wait system vm to comeup before checking timeout, he will make the changes today.

I will close this defect after this, removing blocker and assigning defect to edison
                
      was (Author: rayeesn):
    Yes, i didnt find the issue with latest build 

But as per Edison, this is not a complete fix, we need to wait system vm to comeup before checking timeout, i will make the changes today.

I will close this defect after this, removing blocker and assigning defect to edison
                  
> [Automation]Router and SSVM not coming up after restart 
> --------------------------------------------------------
>
>                 Key: CLOUDSTACK-3538
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3538
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>    Affects Versions: 4.2.0
>         Environment: KVM 
> Build 4.2
>            Reporter: Rayees Namathponnan
>            Assignee: edison su
>             Fix For: 4.2.0
>
>         Attachments: CLOUDSTACK-3538_Debug.rar, CLOUDSTACK-3538.rar
>
>
> Below BVT test cases failed in latest 4.2 run 
> integration.smoke.test_network.TestRebootRouter.test_reboot_router
> integration.smoke.test_network.TestRebootRouter.test_reboot_router
> integration.smoke.test_routers.TestRouterServices.test_08_start_router
> integration.smoke.test_routers.TestRouterServices.test_09_reboot_router 
> integration.smoke.test_ssvm.TestSSVMs.test_07_reboot_ssvm
> All test cases failed to reboot router or SSVM, also observed cloudstack taking more than 20 minute to create new router , 
> Please see the attached log after restart router "r-133-QA"
> -----------------------------------------------------------------------------------
> 1) Issued stop command @ 11:42:02
> 2013-07-15 11:42:02,765 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) Execution is successful.
> 2013-07-15 11:42:02,765 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) Try to stop the vm at first
> 2013-07-15 11:42:10,777 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-1:null) successfully shut down vm r-133-QA
> 2013-07-15 11:42:52,454 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) Ignoring VM r-133-QA in transition state stopping.
> 2013-07-15 11:42:52,455 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) Executing: /usr/share/cloudstack-common/scripts/vm/network/security_group.py get_rule_logs_for_vms
> 2013-07-15 11:42:52,759 DEBUG [kvm.resource.LibvirtComputingResource] (UgentTask-5:null) Execution is successful.
> 2) Started Command issue after 15 minute @ 11:58:03
> 2013-07-15 11:58:03,238 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh netusage.sh 169.254.0.187 -c
> 2013-07-15 11:58:03,291 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) Request:Seq 5-1532821715:  { Cmd , MgmtId: 29066118877352, via: 5, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":133,"name":"r-133-QA","type":"DomainRouter","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"Debian GNU/Linux 5.0 (32-bit)","bootArgs":" template=domP name=r-133-QA eth2ip=10.223.122.92 eth2mask=255.255.255.192 gateway=10.223.122.65 eth0ip=10.1.1.1 eth0mask=255.255.255.0 domain=cs43auto.advanced dhcprange=10.1.1.1 eth1ip=169.254.1.214 eth1mask=255.255.0.0 type=router disable_rp_filter=true dns1=8.8.8.8","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"ee61a6681d35c765","params":{},"uuid":"240b6249-e20f-4c35-93b4-024303bef80a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"5c91d846-6310-4440-aa1d-67b35f2d2c36","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"fff90cb5-06dd-33b3-8815-d78c08ca01d9","id":1,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/rayees/SC_QA_AUTO4/primary","port":2049}},"name":"ROOT-133","size":147456,"path":"d4476933-b42c-45f4-a061-22c1908edca5","volumeId":138,"vmName":"r-133-QA","accountId":67,"format":"QCOW2","id":138}},"diskSeq":0,"type":"ROOT"}],"nics":[{"deviceId":2,"networkRateMbps":200,"defaultNic":true,"uuid":"1b55ad5b-8516-4e79-90a4-33052c43b0e0","ip":"10.223.122.92","netmask":"255.255.255.192","gateway":"10.223.122.65","mac":"06:a0:90:00:00:55","dns1":"8.8.8.8","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1221","isolationUri":"vlan://1221","isSecurityGroupEnabled":false},{"deviceId":0,"networkRateMbps":200,"defaultNic":false,"uuid":"859700c2-124b-4183-a64c-8357f57a7671","ip":"10.1.1.1","netmask":"255.255.255.0","mac":"02:00:21:a1:00:02","dns1":"8.8.8.8","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2311","isolationUri":"vlan://2311","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"4ce6c02e-6df6-4960-857c-3e5fdd2d8e1b","ip":"169.254.1.214","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:d6","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"hostIp":"10.223.50.67","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.214","port":3922,"interval":6,"retries":100,"name":"r-133-QA","wait":0}},{"com.cloud.agent.api.GetDomRVersionCmd":{"accessDetails":{"router.name":"r-133-QA","router.ip":"169.254.1.214"},"wait":0}},{},{"com.cloud.agent.api.routing.IpAssocCommand":{"ipAddresses":[{"accountId":67,"publicIp":"10.223.122.92","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"1221","vlanGateway":"10.223.122.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:92:84:00:00:55","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.name":"r-133-QA","router.ip":"169.254.1.214"},"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"02:00:39:b4:00:01","vmIpAddress":"10.1.1.120","vmName":"dVM","defaultRouter":"10.1.1.1","defaultDns":"10.1.1.1","duid":"00:03:00:01:02:00:39:b4:00:01","isDefault":true,"executeInSequence":false,"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.1.214","router.name":"r-133-QA"},"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"10.1.1.120","vmName":"dVM","executeInSequence":false,"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.name":"r-133-QA","router.ip":"169.254.1.214"},"wait":0}}] }
> 2013-07-15 11:58:03,291 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:null) Processing command: com.cloud.agent.api.StartCommand
> 2013-07-15 11:58:03,363 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null) Execution is successful.
> 3) r-133-QA still in starting state in UI , even after 20 minute
> See the "virsh list" from the host, r-133-QA  is missing
> [root@Rack2Host12 agent]# virsh list
>  Id    Name                           State
> ----------------------------------------------------
>  3     r-130-QA                       running
>  5     r-128-QA                       running
>  6     r-126-QA                       running
>  7     i-65-129-QA                   running
>  10    i-71-139-QA                  running
>  11    r-57-QA                        running
>  13    r-144-QA                      running
> Also below refer below check-in this might be caused this 
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=patch;h=22c6df0ba25f1f1fd3cf489a29ed1f4f4097c89c

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira