You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users-cn@cloudstack.apache.org by 赵涛 <zh...@qq.com> on 2014/07/16 08:31:39 UTC
回复： 回复：RE: 注册模板后出现状态：Storage agent or storage VM disconnected
在ssvm里查看/var/log/cloud/systemvm.log日志 如下：
2014-07-16 06:17:43,730 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: dest ip 10.11.16.0/24 is in the same subnet as eth1 ip 10.11.16.51
2014-07-16 06:17:43,730 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.17.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:17:43,734 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:43,735 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.17.0/24
2014-07-16 06:17:43,735 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 10.11.17.0/24
2014-07-16 06:17:43,737 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:43,737 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 10.11.17.0/24 via 10.11.16.254
2014-07-16 06:17:43,739 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:43,739 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=10.11.17.0/24 via 10.11.16.254
2014-07-16 06:17:43,739 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.24.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:17:43,743 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:43,743 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.24.0/24
2014-07-16 06:17:43,743 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 10.11.24.0/24
2014-07-16 06:17:43,745 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:43,745 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 10.11.24.0/24 via 10.11.16.254
2014-07-16 06:17:43,748 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:43,748 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=10.11.24.0/24 via 10.11.16.254
2014-07-16 06:17:43,748 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 192.168.1.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:17:43,752 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:43,752 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=192.168.1.0/24
2014-07-16 06:17:43,752 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 192.168.1.0/24
2014-07-16 06:17:43,754 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:43,754 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 192.168.1.0/24 via 10.11.16.254
2014-07-16 06:17:43,756 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:43,756 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=192.168.1.0/24 via 10.11.16.254
2014-07-16 06:17:43,756 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Executing: /usr/local/cloud/systemvm/config_auth.sh cloud nD7iafqpzcsxqme
2014-07-16 06:17:43,762 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:43,762 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Adding password for user cloud
2014-07-16 06:17:43,763 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Seq 46-101646339: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"Answer":{"result":true,"details":"","wait":0}}] }
2014-07-16 06:17:43,832 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Request:Seq 46-101646340: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"storage.ListTemplateCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] }
2014-07-16 06:17:43,832 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Processing command: com.cloud.agent.api.storage.ListTemplateCommand
2014-07-16 06:17:43,832 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: mount
2014-07-16 06:17:43,834 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:17:43,836 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/
2014-07-16 06:17:44,154 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:17:44,154 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) found 28 templates[/mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/4/e970d345-e30b-30c4-879c-88d63200a721.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/3/6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/218/d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/254/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/209/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/216/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/264/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/261/c90f1ba1-8610-3dbb-bb6e-2bcad6ff75be.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/217/3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/287/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/280/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/273/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/249/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/258/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/263/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/283/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/267/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/282/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/208/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/265/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/215/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/210/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/211/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/262/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/268/c036ba8f-7976-342b-a82a-2fb58d5724ac.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/252/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/212/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/221/221-2-3706787c-d305-39e0-9680-9f84359df07a.iso]
2014-07-16 06:17:44,156 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: centos55-x86_64, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/4/e970d345-e30b-30c4-879c-88d63200a721.qcow2
2014-07-16 06:17:44,158 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: routing-3, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/3/6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2
2014-07-16 06:17:44,160 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: 218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/218/d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2
2014-07-16 06:17:44,162 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/254
2014-07-16 06:17:44,169 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/209
2014-07-16 06:17:44,175 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/216
2014-07-16 06:17:44,182 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/264
2014-07-16 06:17:44,188 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/261
2014-07-16 06:17:44,194 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: 217-2-0245468c-5e25-3513-92a4-89b4a045751f, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/217/3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2
2014-07-16 06:17:44,196 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/287
2014-07-16 06:17:44,202 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/280
2014-07-16 06:17:44,208 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/273
2014-07-16 06:17:44,214 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/249
2014-07-16 06:17:44,220 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/258
2014-07-16 06:17:44,226 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/263
2014-07-16 06:17:44,231 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/283
2014-07-16 06:17:44,237 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/267
2014-07-16 06:17:44,243 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/282
2014-07-16 06:17:44,248 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/208
2014-07-16 06:17:44,254 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/265
2014-07-16 06:17:44,260 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/215
2014-07-16 06:17:44,266 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/210
2014-07-16 06:17:44,272 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/211
2014-07-16 06:17:44,277 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/262
2014-07-16 06:17:44,284 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/268
2014-07-16 06:17:44,290 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/252
2014-07-16 06:17:44,296 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/212
2014-07-16 06:17:44,303 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: 221-2-3706787c-d305-39e0-9680-9f84359df07a, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/221/221-2-3706787c-d305-39e0-9680-9f84359df07a.iso
2014-07-16 06:17:44,304 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Seq 46-101646340: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"storage.ListTemplateAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{"221-2-3706787c-d305-39e0-9680-9f84359df07a":{"templateName":"221-2-3706787c-d305-39e0-9680-9f84359df07a","installPath":"template/tmpl/2/221//221-2-3706787c-d305-39e0-9680-9f84359df07a.iso","size":45762560,"physicalSize":45762560,"id":221,"isPublic":true,"isCorrupted":false},"217-2-0245468c-5e25-3513-92a4-89b4a045751f":{"templateName":"217-2-0245468c-5e25-3513-92a4-89b4a045751f","installPath":"template/tmpl/2/217//3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2","size":107374182400,"physicalSize":4641128448,"id":217,"isPublic":true,"isCorrupted":false},"routing-3":{"templateName":"routing-3","installPath":"template/tmpl/1/3//6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2","size":725811200,"physicalSize":725811200,"id":3,"isPublic":true,"isCorrupted":false},"centos55-x86_64":{"templateName":"centos55-x86_64","installPath":"template/tmpl/1/4//e970d345-e30b-30c4-879c-88d63200a721.qcow2","size":8589934592,"physicalSize":1769537536,"id":4,"isPublic":true,"isCorrupted":false},"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1":{"templateName":"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1","installPath":"template/tmpl/2/218//d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2","size":107374182400,"physicalSize":15100084224,"id":218,"isPublic":true,"isCorrupted":false}},"result":true,"details":"success","wait":0}}] }
2014-07-16 06:17:44,676 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Request:Seq 46-101646341: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"storage.ListVolumeCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] }
2014-07-16 06:17:44,676 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Processing command: com.cloud.agent.api.storage.ListVolumeCommand
2014-07-16 06:17:44,677 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:) Executing: mount
2014-07-16 06:17:44,680 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:) Execution is successful.
2014-07-16 06:17:44,681 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvolume.sh -r /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/volumes/
2014-07-16 06:17:44,691 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Execution is successful.
2014-07-16 06:17:44,691 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) found 0 volumes[]
2014-07-16 06:17:44,691 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Seq 46-101646341: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"storage.ListVolumeAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{},"result":true,"details":"success","wait":0}}] }
2014-07-16 06:17:44,709 DEBUG [utils.nio.NioConnection] (Agent-Selector:) Location 1: Socket Socket[addr=/10.11.24.4,port=8250,localport=50806] closed on read. Probably -1 returned: Connection closed with -1 on reading size.
2014-07-16 06:17:44,710 DEBUG [utils.nio.NioConnection] (Agent-Selector:) Closing socket Socket[addr=/10.11.24.4,port=8250,localport=50806]
2014-07-16 06:17:44,710 DEBUG [cloud.agent.Agent] (Agent-Handler-4:) Clearing watch list: 1
2014-07-16 06:17:46,179 INFO [cloud.agent.Agent] (Agent-Handler-5:) Connected to the server
2014-07-16 06:17:49,710 INFO [cloud.agent.Agent] (Agent-Handler-4:) Lost connection to the server. Dealing with the remaining commands...
2014-07-16 06:17:54,712 INFO [cloud.agent.Agent] (Agent-Handler-4:) Reconnecting...
2014-07-16 06:17:54,712 INFO [utils.nio.NioClient] (Agent-Selector:) Connecting to 10.11.24.4:8250
2014-07-16 06:17:54,802 INFO [utils.nio.NioClient] (Agent-Selector:) SSL: Handshake done
2014-07-16 06:17:54,803 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for private nic: 10.11.16.51 - 06:5e:88:00:00:01-255.255.255.0
2014-07-16 06:17:54,803 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for storage nic: 10.11.16.51 - 06:5e:88:00:00:01-255.255.255.0
2014-07-16 06:17:54,803 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for pubic nic: 10.11.17.32 - 06:c4:0c:00:00:48-255.255.255.0
2014-07-16 06:17:54,803 DEBUG [storage.resource.NfsSecondaryStorageResource] (Agent-Handler-1:) Executing: /bin/bash -c ln -sf /mnt/SecStorage /var/www/html/copy
2014-07-16 06:17:54,806 DEBUG [storage.resource.NfsSecondaryStorageResource] (Agent-Handler-1:) Execution is successful.
2014-07-16 06:17:54,806 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Executing: hostname
2014-07-16 06:17:54,807 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Execution is successful.
2014-07-16 06:17:54,808 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Sending Startup: Seq 46-328: { Cmd , MgmtId: -1, via: 46, Ver: v1, Flags: 101, [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"2","pod":"2","guid":"s-914-VM-NfsSecondaryStorageResource","name":"s-914-VM","id":46,"version":"2.2.20130913151423","iqn":"NoIqn","publicIpAddress":"10.11.17.32","publicNetmask":"255.255.255.0","publicMacAddress":"06:c4:0c:00:00:48","privateIpAddress":"10.11.16.51","privateMacAddress":"06:5e:88:00:00:01","privateNetmask":"255.255.255.0","storageIpAddress":"10.11.16.51","storageNetmask":"255.255.255.0","storageMacAddress":"06:5e:88:00:00:01","resourceName":"NfsSecondaryStorageResource","wait":0}}] }
2014-07-16 06:17:54,808 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Startup task created
2014-07-16 06:17:55,007 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Received response: Seq 46-328: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100100, [{"StartupAnswer":{"hostId":46,"pingInterval":60,"result":true,"wait":0}}] }
2014-07-16 06:17:55,007 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Startup task cancelled
2014-07-16 06:17:55,008 INFO [cloud.agent.Agent] (Agent-Handler-2:) Proccess agent startup answer, agent id = 46
2014-07-16 06:17:55,008 INFO [cloud.agent.Agent] (Agent-Handler-2:) Set agent id 46
2014-07-16 06:17:55,008 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Adding a watch list
2014-07-16 06:17:55,008 INFO [cloud.agent.Agent] (Agent-Handler-2:) Startup Response Received: agent id = 46
2014-07-16 06:17:55,008 DEBUG [cloud.agent.Agent] (UgentTask-5:) Sending ping: Seq 46-329: { Cmd , MgmtId: -1, via: 46, Ver: v1, Flags: 11, [{"PingStorageCommand":{"changes":{},"hostType":"Storage","hostId":46,"wait":0}}] }
2014-07-16 06:17:55,019 DEBUG [cloud.agent.Agent] (Agent-Handler-3:) Received response: Seq 46-329: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100010, [{"PingAnswer":{"_command":{"hostType":"Storage","hostId":46,"wait":0},"result":true,"wait":0}}] }
2014-07-16 06:17:55,020 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Request:Seq 46-1303117825: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 10100111, [{"SecStorageSetupCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","certs":{},"wait":0}}] }
2014-07-16 06:17:55,021 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Processing command: com.cloud.agent.api.SecStorageSetupCommand
2014-07-16 06:17:55,021 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.80, eth1mask=255.255.255.0,destIp=10.11.18.7
2014-07-16 06:17:55,021 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: /bin/bash -c ip route delete 10.11.18.7
2014-07-16 06:17:55,023 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:17:55,023 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: /bin/bash -c ip route add 10.11.18.7 via 10.11.16.254
2014-07-16 06:17:55,025 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:17:55,025 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) addRouteToInternalIp: added route to internal ip=10.11.18.7 via 10.11.16.254
2014-07-16 06:17:55,026 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: mount
2014-07-16 06:17:55,027 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:17:55,027 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Executing: /usr/local/cloud/systemvm/config_ssl.sh -i 10.11.17.32 -h s-914-VM -k /tmp/prvkey7579279569296277166.tmp -p /tmp/pubcert3403794863871149570.tmp
2014-07-16 06:17:57,188 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:17:57,188 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Stopping web server: apache2apache2: Could not reliably determine the server's fully qualified domain name, using 10.11.17.32 for ServerName
... waiting ..
Starting web server: apache2apache2: Could not reliably determine the server's fully qualified domain name, using 10.11.17.32 for ServerName
.
2014-07-16 06:17:57,188 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Seq 46-1303117825: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"SecStorageSetupAnswer":{"_dir":"a35b9ab9-d061-3ed7-bbe7-82022eec78e4","result":true,"details":"success","wait":0}}] }
2014-07-16 06:17:57,290 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Request:Seq 46-1303117826: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100011, [{"SecStorageFirewallCfgCommand":{"isAppendAIp":false,"wait":0}}] }
2014-07-16 06:17:57,290 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Processing command: com.cloud.agent.api.SecStorageFirewallCfgCommand
2014-07-16 06:17:57,290 DEBUG [utils.script.Script] (agentRequest-Handler-4:) Executing: /usr/local/cloud/systemvm/ipfirewall.sh false
2014-07-16 06:17:57,294 DEBUG [utils.script.Script] (agentRequest-Handler-4:) Execution is successful.
2014-07-16 06:17:57,294 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Seq 46-1303117826: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}] }
2014-07-16 06:17:57,364 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Request:Seq 46-1303117827: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"SecStorageVMSetupCommand":{"allowedInternalSites":["10.11.18.0/24","10.11.19.0/24","10.11.16.0/24","10.11.17.0/24","10.11.24.0/24","192.168.1.0/24"],"copyUserName":"cloud","copyPassword":"nD7iafqpzcsxqme","wait":0}}] }
2014-07-16 06:17:57,364 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Processing command: com.cloud.agent.api.SecStorageVMSetupCommand
2014-07-16 06:17:57,364 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.18.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:17:57,369 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,369 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.18.0/24
2014-07-16 06:17:57,369 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 10.11.18.0/24
2014-07-16 06:17:57,371 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,371 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 10.11.18.0/24 via 10.11.16.254
2014-07-16 06:17:57,373 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,374 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=10.11.18.0/24 via 10.11.16.254
2014-07-16 06:17:57,374 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.19.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:17:57,378 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,378 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.19.0/24
2014-07-16 06:17:57,378 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 10.11.19.0/24
2014-07-16 06:17:57,380 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,380 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 10.11.19.0/24 via 10.11.16.254
2014-07-16 06:17:57,382 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,382 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=10.11.19.0/24 via 10.11.16.254
2014-07-16 06:17:57,382 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.16.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:17:57,386 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,386 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.16.0/24
2014-07-16 06:17:57,386 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: dest ip 10.11.16.0/24 is in the same subnet as eth1 ip 10.11.16.51
2014-07-16 06:17:57,386 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.17.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:17:57,390 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,390 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.17.0/24
2014-07-16 06:17:57,390 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 10.11.17.0/24
2014-07-16 06:17:57,392 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,393 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 10.11.17.0/24 via 10.11.16.254
2014-07-16 06:17:57,395 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,395 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=10.11.17.0/24 via 10.11.16.254
2014-07-16 06:17:57,395 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.24.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:17:57,399 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,399 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.24.0/24
2014-07-16 06:17:57,399 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 10.11.24.0/24
2014-07-16 06:17:57,401 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,401 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 10.11.24.0/24 via 10.11.16.254
2014-07-16 06:17:57,403 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,403 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=10.11.24.0/24 via 10.11.16.254
2014-07-16 06:17:57,403 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 192.168.1.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:17:57,407 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,407 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=192.168.1.0/24
2014-07-16 06:17:57,407 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 192.168.1.0/24
2014-07-16 06:17:57,409 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,409 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 192.168.1.0/24 via 10.11.16.254
2014-07-16 06:17:57,411 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,412 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=192.168.1.0/24 via 10.11.16.254
2014-07-16 06:17:57,412 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Executing: /usr/local/cloud/systemvm/config_auth.sh cloud nD7iafqpzcsxqme
2014-07-16 06:17:57,417 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:17:57,417 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Adding password for user cloud
2014-07-16 06:17:57,417 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Seq 46-1303117827: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"Answer":{"result":true,"details":"","wait":0}}] }
2014-07-16 06:17:57,481 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Request:Seq 46-1303117828: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"storage.ListTemplateCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] }
2014-07-16 06:17:57,481 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Processing command: com.cloud.agent.api.storage.ListTemplateCommand
2014-07-16 06:17:57,482 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: mount
2014-07-16 06:17:57,484 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:17:57,485 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/
2014-07-16 06:17:57,765 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:17:57,765 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) found 28 templates[/mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/4/e970d345-e30b-30c4-879c-88d63200a721.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/3/6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/218/d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/254/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/209/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/216/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/264/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/261/c90f1ba1-8610-3dbb-bb6e-2bcad6ff75be.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/217/3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/287/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/280/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/273/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/249/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/258/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/263/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/283/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/267/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/282/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/208/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/265/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/215/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/210/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/211/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/262/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/268/c036ba8f-7976-342b-a82a-2fb58d5724ac.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/252/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/212/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/221/221-2-3706787c-d305-39e0-9680-9f84359df07a.iso]
2014-07-16 06:17:57,767 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: centos55-x86_64, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/4/e970d345-e30b-30c4-879c-88d63200a721.qcow2
2014-07-16 06:17:57,770 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: routing-3, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/3/6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2
2014-07-16 06:17:57,772 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: 218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/218/d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2
2014-07-16 06:17:57,774 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/254
2014-07-16 06:17:57,780 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/209
2014-07-16 06:17:57,785 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/216
2014-07-16 06:17:57,790 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/264
2014-07-16 06:17:57,796 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/261
2014-07-16 06:17:57,803 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: 217-2-0245468c-5e25-3513-92a4-89b4a045751f, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/217/3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2
2014-07-16 06:17:57,805 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/287
2014-07-16 06:17:57,811 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/280
2014-07-16 06:17:57,816 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/273
2014-07-16 06:17:57,821 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/249
2014-07-16 06:17:57,827 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/258
2014-07-16 06:17:57,832 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/263
2014-07-16 06:17:57,838 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/283
2014-07-16 06:17:57,843 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/267
2014-07-16 06:17:57,849 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/282
2014-07-16 06:17:57,855 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/208
2014-07-16 06:17:57,861 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/265
2014-07-16 06:17:57,867 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/215
2014-07-16 06:17:57,872 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/210
2014-07-16 06:17:57,878 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/211
2014-07-16 06:17:57,883 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/262
2014-07-16 06:17:57,889 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/268
2014-07-16 06:17:57,894 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/252
2014-07-16 06:17:57,899 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/212
2014-07-16 06:17:57,905 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: 221-2-3706787c-d305-39e0-9680-9f84359df07a, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/221/221-2-3706787c-d305-39e0-9680-9f84359df07a.iso
2014-07-16 06:17:57,906 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Seq 46-1303117828: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"storage.ListTemplateAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{"221-2-3706787c-d305-39e0-9680-9f84359df07a":{"templateName":"221-2-3706787c-d305-39e0-9680-9f84359df07a","installPath":"template/tmpl/2/221//221-2-3706787c-d305-39e0-9680-9f84359df07a.iso","size":45762560,"physicalSize":45762560,"id":221,"isPublic":true,"isCorrupted":false},"217-2-0245468c-5e25-3513-92a4-89b4a045751f":{"templateName":"217-2-0245468c-5e25-3513-92a4-89b4a045751f","installPath":"template/tmpl/2/217//3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2","size":107374182400,"physicalSize":4641128448,"id":217,"isPublic":true,"isCorrupted":false},"routing-3":{"templateName":"routing-3","installPath":"template/tmpl/1/3//6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2","size":725811200,"physicalSize":725811200,"id":3,"isPublic":true,"isCorrupted":false},"centos55-x86_64":{"templateName":"centos55-x86_64","installPath":"template/tmpl/1/4//e970d345-e30b-30c4-879c-88d63200a721.qcow2","size":8589934592,"physicalSize":1769537536,"id":4,"isPublic":true,"isCorrupted":false},"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1":{"templateName":"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1","installPath":"template/tmpl/2/218//d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2","size":107374182400,"physicalSize":15100084224,"id":218,"isPublic":true,"isCorrupted":false}},"result":true,"details":"success","wait":0}}] }
2014-07-16 06:17:58,225 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Request:Seq 46-1303117829: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"storage.ListVolumeCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] }
2014-07-16 06:17:58,226 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Processing command: com.cloud.agent.api.storage.ListVolumeCommand
2014-07-16 06:17:58,226 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:) Executing: mount
2014-07-16 06:17:58,228 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:) Execution is successful.
2014-07-16 06:17:58,229 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvolume.sh -r /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/volumes/
2014-07-16 06:17:58,238 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Execution is successful.
2014-07-16 06:17:58,238 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) found 0 volumes[]
2014-07-16 06:17:58,238 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Seq 46-1303117829: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"storage.ListVolumeAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{},"result":true,"details":"success","wait":0}}] }
2014-07-16 06:17:58,269 DEBUG [utils.nio.NioConnection] (Agent-Selector:) Location 1: Socket Socket[addr=/10.11.24.4,port=8250,localport=50807] closed on read. Probably -1 returned: Connection closed with -1 on reading size.
2014-07-16 06:17:58,269 DEBUG [utils.nio.NioConnection] (Agent-Selector:) Closing socket Socket[addr=/10.11.24.4,port=8250,localport=50807]
2014-07-16 06:17:58,269 DEBUG [cloud.agent.Agent] (Agent-Handler-4:) Clearing watch list: 1
2014-07-16 06:17:59,803 INFO [cloud.agent.Agent] (Agent-Handler-4:) Connected to the server
2014-07-16 06:18:03,270 INFO [cloud.agent.Agent] (Agent-Handler-4:) Lost connection to the server. Dealing with the remaining commands...
2014-07-16 06:18:08,271 INFO [cloud.agent.Agent] (Agent-Handler-4:) Reconnecting...
2014-07-16 06:18:08,271 INFO [utils.nio.NioClient] (Agent-Selector:) Connecting to 10.11.24.4:8250
2014-07-16 06:18:08,362 INFO [utils.nio.NioClient] (Agent-Selector:) SSL: Handshake done
2014-07-16 06:18:08,363 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for private nic: 10.11.16.51 - 06:5e:88:00:00:01-255.255.255.0
2014-07-16 06:18:08,363 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for storage nic: 10.11.16.51 - 06:5e:88:00:00:01-255.255.255.0
2014-07-16 06:18:08,363 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for pubic nic: 10.11.17.32 - 06:c4:0c:00:00:48-255.255.255.0
2014-07-16 06:18:08,363 DEBUG [storage.resource.NfsSecondaryStorageResource] (Agent-Handler-1:) Executing: /bin/bash -c ln -sf /mnt/SecStorage /var/www/html/copy
2014-07-16 06:18:08,365 DEBUG [storage.resource.NfsSecondaryStorageResource] (Agent-Handler-1:) Execution is successful.
2014-07-16 06:18:08,366 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Executing: hostname
2014-07-16 06:18:08,367 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Execution is successful.
2014-07-16 06:18:08,367 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Sending Startup: Seq 46-330: { Cmd , MgmtId: -1, via: 46, Ver: v1, Flags: 101, [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"2","pod":"2","guid":"s-914-VM-NfsSecondaryStorageResource","name":"s-914-VM","id":46,"version":"2.2.20130913151423","iqn":"NoIqn","publicIpAddress":"10.11.17.32","publicNetmask":"255.255.255.0","publicMacAddress":"06:c4:0c:00:00:48","privateIpAddress":"10.11.16.51","privateMacAddress":"06:5e:88:00:00:01","privateNetmask":"255.255.255.0","storageIpAddress":"10.11.16.51","storageNetmask":"255.255.255.0","storageMacAddress":"06:5e:88:00:00:01","resourceName":"NfsSecondaryStorageResource","wait":0}}] }
2014-07-16 06:18:08,367 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Startup task created
2014-07-16 06:18:08,567 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Received response: Seq 46-330: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100100, [{"StartupAnswer":{"hostId":46,"pingInterval":60,"result":true,"wait":0}}] }
2014-07-16 06:18:08,568 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Startup task cancelled
2014-07-16 06:18:08,568 INFO [cloud.agent.Agent] (Agent-Handler-2:) Proccess agent startup answer, agent id = 46
2014-07-16 06:18:08,568 INFO [cloud.agent.Agent] (Agent-Handler-2:) Set agent id 46
2014-07-16 06:18:08,568 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Adding a watch list
2014-07-16 06:18:08,568 INFO [cloud.agent.Agent] (Agent-Handler-2:) Startup Response Received: agent id = 46
2014-07-16 06:18:08,568 DEBUG [cloud.agent.Agent] (UgentTask-5:) Sending ping: Seq 46-331: { Cmd , MgmtId: -1, via: 46, Ver: v1, Flags: 11, [{"PingStorageCommand":{"changes":{},"hostType":"Storage","hostId":46,"wait":0}}] }
2014-07-16 06:18:08,571 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Request:Seq 46-1683816449: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 10100111, [{"SecStorageSetupCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","certs":{},"wait":0}}] }
2014-07-16 06:18:08,571 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Processing command: com.cloud.agent.api.SecStorageSetupCommand
2014-07-16 06:18:08,571 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.80, eth1mask=255.255.255.0,destIp=10.11.18.7
2014-07-16 06:18:08,571 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: /bin/bash -c ip route delete 10.11.18.7
2014-07-16 06:18:08,574 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:08,574 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: /bin/bash -c ip route add 10.11.18.7 via 10.11.16.254
2014-07-16 06:18:08,576 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:08,576 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) addRouteToInternalIp: added route to internal ip=10.11.18.7 via 10.11.16.254
2014-07-16 06:18:08,576 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: mount
2014-07-16 06:18:08,578 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:08,578 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Executing: /usr/local/cloud/systemvm/config_ssl.sh -i 10.11.17.32 -h s-914-VM -k /tmp/prvkey896557704488795261.tmp -p /tmp/pubcert6295364737312068298.tmp
2014-07-16 06:18:08,615 DEBUG [cloud.agent.Agent] (Agent-Handler-4:) Received response: Seq 46-331: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100010, [{"PingAnswer":{"_command":{"hostType":"Storage","hostId":46,"wait":0},"result":true,"wait":0}}] }
2014-07-16 06:18:09,780 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:09,781 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Stopping web server: apache2apache2: Could not reliably determine the server's fully qualified domain name, using 10.11.17.32 for ServerName
... waiting .
Starting web server: apache2apache2: Could not reliably determine the server's fully qualified domain name, using 10.11.17.32 for ServerName
.
2014-07-16 06:18:09,781 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Seq 46-1683816449: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"SecStorageSetupAnswer":{"_dir":"a35b9ab9-d061-3ed7-bbe7-82022eec78e4","result":true,"details":"success","wait":0}}] }
2014-07-16 06:18:09,875 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Request:Seq 46-1683816450: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100011, [{"SecStorageFirewallCfgCommand":{"isAppendAIp":false,"wait":0}}] }
2014-07-16 06:18:09,875 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Processing command: com.cloud.agent.api.SecStorageFirewallCfgCommand
2014-07-16 06:18:09,875 DEBUG [utils.script.Script] (agentRequest-Handler-4:) Executing: /usr/local/cloud/systemvm/ipfirewall.sh false
2014-07-16 06:18:09,879 DEBUG [utils.script.Script] (agentRequest-Handler-4:) Execution is successful.
2014-07-16 06:18:09,879 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Seq 46-1683816450: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}] }
2014-07-16 06:18:09,948 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Request:Seq 46-1683816451: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"SecStorageVMSetupCommand":{"allowedInternalSites":["10.11.18.0/24","10.11.19.0/24","10.11.16.0/24","10.11.17.0/24","10.11.24.0/24","192.168.1.0/24"],"copyUserName":"cloud","copyPassword":"nD7iafqpzcsxqme","wait":0}}] }
2014-07-16 06:18:09,948 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Processing command: com.cloud.agent.api.SecStorageVMSetupCommand
2014-07-16 06:18:09,948 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.18.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:09,952 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,952 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.18.0/24
2014-07-16 06:18:09,952 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 10.11.18.0/24
2014-07-16 06:18:09,955 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,955 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 10.11.18.0/24 via 10.11.16.254
2014-07-16 06:18:09,957 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,957 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=10.11.18.0/24 via 10.11.16.254
2014-07-16 06:18:09,957 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.19.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:09,961 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,961 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.19.0/24
2014-07-16 06:18:09,961 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 10.11.19.0/24
2014-07-16 06:18:09,963 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,963 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 10.11.19.0/24 via 10.11.16.254
2014-07-16 06:18:09,965 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,965 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=10.11.19.0/24 via 10.11.16.254
2014-07-16 06:18:09,965 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.16.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:09,969 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,970 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.16.0/24
2014-07-16 06:18:09,970 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: dest ip 10.11.16.0/24 is in the same subnet as eth1 ip 10.11.16.51
2014-07-16 06:18:09,970 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.17.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:09,974 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,974 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.17.0/24
2014-07-16 06:18:09,974 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 10.11.17.0/24
2014-07-16 06:18:09,976 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,977 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 10.11.17.0/24 via 10.11.16.254
2014-07-16 06:18:09,979 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,979 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=10.11.17.0/24 via 10.11.16.254
2014-07-16 06:18:09,979 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.24.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:09,983 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,983 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.24.0/24
2014-07-16 06:18:09,983 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 10.11.24.0/24
2014-07-16 06:18:09,985 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,985 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 10.11.24.0/24 via 10.11.16.254
2014-07-16 06:18:09,987 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,987 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=10.11.24.0/24 via 10.11.16.254
2014-07-16 06:18:09,987 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 192.168.1.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:09,991 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,991 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=192.168.1.0/24
2014-07-16 06:18:09,991 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route delete 192.168.1.0/24
2014-07-16 06:18:09,993 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,993 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Executing: /bin/bash -c ip route add 192.168.1.0/24 via 10.11.16.254
2014-07-16 06:18:09,995 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:09,995 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:) addRouteToInternalIp: added route to internal ip=192.168.1.0/24 via 10.11.16.254
2014-07-16 06:18:09,995 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Executing: /usr/local/cloud/systemvm/config_auth.sh cloud nD7iafqpzcsxqme
2014-07-16 06:18:10,001 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:10,001 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Adding password for user cloud
2014-07-16 06:18:10,001 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Seq 46-1683816451: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"Answer":{"result":true,"details":"","wait":0}}] }
2014-07-16 06:18:10,066 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Request:Seq 46-1683816452: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"storage.ListTemplateCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] }
2014-07-16 06:18:10,066 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Processing command: com.cloud.agent.api.storage.ListTemplateCommand
2014-07-16 06:18:10,067 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: mount
2014-07-16 06:18:10,068 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:10,070 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/
2014-07-16 06:18:10,349 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:10,349 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) found 28 templates[/mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/4/e970d345-e30b-30c4-879c-88d63200a721.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/3/6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/218/d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/254/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/209/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/216/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/264/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/261/c90f1ba1-8610-3dbb-bb6e-2bcad6ff75be.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/217/3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/287/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/280/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/273/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/249/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/258/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/263/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/283/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/267/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/282/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/208/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/265/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/215/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/210/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/211/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/262/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/268/c036ba8f-7976-342b-a82a-2fb58d5724ac.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/252/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/212/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/221/221-2-3706787c-d305-39e0-9680-9f84359df07a.iso]
2014-07-16 06:18:10,351 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: centos55-x86_64, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/4/e970d345-e30b-30c4-879c-88d63200a721.qcow2
2014-07-16 06:18:10,353 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: routing-3, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/3/6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2
2014-07-16 06:18:10,355 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: 218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/218/d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2
2014-07-16 06:18:10,357 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/254
2014-07-16 06:18:10,363 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/209
2014-07-16 06:18:10,369 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/216
2014-07-16 06:18:10,374 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/264
2014-07-16 06:18:10,380 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/261
2014-07-16 06:18:10,385 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: 217-2-0245468c-5e25-3513-92a4-89b4a045751f, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/217/3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2
2014-07-16 06:18:10,387 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/287
2014-07-16 06:18:10,393 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/280
2014-07-16 06:18:10,399 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/273
2014-07-16 06:18:10,404 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/249
2014-07-16 06:18:10,410 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/258
2014-07-16 06:18:10,416 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/263
2014-07-16 06:18:10,421 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/283
2014-07-16 06:18:10,427 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/267
2014-07-16 06:18:10,432 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/282
2014-07-16 06:18:10,438 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/208
2014-07-16 06:18:10,443 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/265
2014-07-16 06:18:10,449 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/215
2014-07-16 06:18:10,455 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/210
2014-07-16 06:18:10,460 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/211
2014-07-16 06:18:10,466 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/262
2014-07-16 06:18:10,471 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/268
2014-07-16 06:18:10,477 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/252
2014-07-16 06:18:10,483 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/212
2014-07-16 06:18:10,488 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-3:) Added template name: 221-2-3706787c-d305-39e0-9680-9f84359df07a, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/221/221-2-3706787c-d305-39e0-9680-9f84359df07a.iso
2014-07-16 06:18:10,489 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Seq 46-1683816452: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"storage.ListTemplateAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{"221-2-3706787c-d305-39e0-9680-9f84359df07a":{"templateName":"221-2-3706787c-d305-39e0-9680-9f84359df07a","installPath":"template/tmpl/2/221//221-2-3706787c-d305-39e0-9680-9f84359df07a.iso","size":45762560,"physicalSize":45762560,"id":221,"isPublic":true,"isCorrupted":false},"217-2-0245468c-5e25-3513-92a4-89b4a045751f":{"templateName":"217-2-0245468c-5e25-3513-92a4-89b4a045751f","installPath":"template/tmpl/2/217//3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2","size":107374182400,"physicalSize":4641128448,"id":217,"isPublic":true,"isCorrupted":false},"routing-3":{"templateName":"routing-3","installPath":"template/tmpl/1/3//6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2","size":725811200,"physicalSize":725811200,"id":3,"isPublic":true,"isCorrupted":false},"centos55-x86_64":{"templateName":"centos55-x86_64","installPath":"template/tmpl/1/4//e970d345-e30b-30c4-879c-88d63200a721.qcow2","size":8589934592,"physicalSize":1769537536,"id":4,"isPublic":true,"isCorrupted":false},"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1":{"templateName":"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1","installPath":"template/tmpl/2/218//d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2","size":107374182400,"physicalSize":15100084224,"id":218,"isPublic":true,"isCorrupted":false}},"result":true,"details":"success","wait":0}}] }
2014-07-16 06:18:10,823 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Request:Seq 46-1683816453: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"storage.ListVolumeCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] }
2014-07-16 06:18:10,823 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Processing command: com.cloud.agent.api.storage.ListVolumeCommand
2014-07-16 06:18:10,824 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:) Executing: mount
2014-07-16 06:18:10,826 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:) Execution is successful.
2014-07-16 06:18:10,827 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvolume.sh -r /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/volumes/
2014-07-16 06:18:10,836 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Execution is successful.
2014-07-16 06:18:10,836 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) found 0 volumes[]
2014-07-16 06:18:10,836 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Seq 46-1683816453: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"storage.ListVolumeAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{},"result":true,"details":"success","wait":0}}] }
2014-07-16 06:18:10,855 DEBUG [utils.nio.NioConnection] (Agent-Selector:) Location 1: Socket Socket[addr=/10.11.24.4,port=8250,localport=50808] closed on read. Probably -1 returned: Connection closed with -1 on reading size.
2014-07-16 06:18:10,856 DEBUG [utils.nio.NioConnection] (Agent-Selector:) Closing socket Socket[addr=/10.11.24.4,port=8250,localport=50808]
2014-07-16 06:18:10,856 DEBUG [cloud.agent.Agent] (Agent-Handler-4:) Clearing watch list: 1
2014-07-16 06:18:13,363 INFO [cloud.agent.Agent] (Agent-Handler-4:) Connected to the server
2014-07-16 06:18:15,856 INFO [cloud.agent.Agent] (Agent-Handler-4:) Lost connection to the server. Dealing with the remaining commands...
2014-07-16 06:18:20,858 INFO [cloud.agent.Agent] (Agent-Handler-4:) Reconnecting...
2014-07-16 06:18:20,858 INFO [utils.nio.NioClient] (Agent-Selector:) Connecting to 10.11.24.4:8250
2014-07-16 06:18:20,946 INFO [utils.nio.NioClient] (Agent-Selector:) SSL: Handshake done
2014-07-16 06:18:20,947 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for private nic: 10.11.16.51 - 06:5e:88:00:00:01-255.255.255.0
2014-07-16 06:18:20,947 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for storage nic: 10.11.16.51 - 06:5e:88:00:00:01-255.255.255.0
2014-07-16 06:18:20,948 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for pubic nic: 10.11.17.32 - 06:c4:0c:00:00:48-255.255.255.0
2014-07-16 06:18:20,948 DEBUG [storage.resource.NfsSecondaryStorageResource] (Agent-Handler-1:) Executing: /bin/bash -c ln -sf /mnt/SecStorage /var/www/html/copy
2014-07-16 06:18:20,950 DEBUG [storage.resource.NfsSecondaryStorageResource] (Agent-Handler-1:) Execution is successful.
2014-07-16 06:18:20,950 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Executing: hostname
2014-07-16 06:18:20,951 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Execution is successful.
2014-07-16 06:18:20,952 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Sending Startup: Seq 46-332: { Cmd , MgmtId: -1, via: 46, Ver: v1, Flags: 101, [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"2","pod":"2","guid":"s-914-VM-NfsSecondaryStorageResource","name":"s-914-VM","id":46,"version":"2.2.20130913151423","iqn":"NoIqn","publicIpAddress":"10.11.17.32","publicNetmask":"255.255.255.0","publicMacAddress":"06:c4:0c:00:00:48","privateIpAddress":"10.11.16.51","privateMacAddress":"06:5e:88:00:00:01","privateNetmask":"255.255.255.0","storageIpAddress":"10.11.16.51","storageNetmask":"255.255.255.0","storageMacAddress":"06:5e:88:00:00:01","resourceName":"NfsSecondaryStorageResource","wait":0}}] }
2014-07-16 06:18:20,952 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Startup task created
2014-07-16 06:18:21,167 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Received response: Seq 46-332: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100100, [{"StartupAnswer":{"hostId":46,"pingInterval":60,"result":true,"wait":0}}] }
2014-07-16 06:18:21,167 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Startup task cancelled
2014-07-16 06:18:21,167 INFO [cloud.agent.Agent] (Agent-Handler-2:) Proccess agent startup answer, agent id = 46
2014-07-16 06:18:21,167 INFO [cloud.agent.Agent] (Agent-Handler-2:) Set agent id 46
2014-07-16 06:18:21,167 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Adding a watch list
2014-07-16 06:18:21,168 INFO [cloud.agent.Agent] (Agent-Handler-2:) Startup Response Received: agent id = 46
2014-07-16 06:18:21,168 DEBUG [cloud.agent.Agent] (UgentTask-5:) Sending ping: Seq 46-333: { Cmd , MgmtId: -1, via: 46, Ver: v1, Flags: 11, [{"PingStorageCommand":{"changes":{},"hostType":"Storage","hostId":46,"wait":0}}] }
2014-07-16 06:18:21,184 DEBUG [cloud.agent.Agent] (Agent-Handler-3:) Received response: Seq 46-333: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100010, [{"PingAnswer":{"_command":{"hostType":"Storage","hostId":46,"wait":0},"result":true,"wait":0}}] }
2014-07-16 06:18:21,188 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Request:Seq 46-1933115393: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 10100111, [{"SecStorageSetupCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","certs":{},"wait":0}}] }
2014-07-16 06:18:21,188 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Processing command: com.cloud.agent.api.SecStorageSetupCommand
2014-07-16 06:18:21,188 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.80, eth1mask=255.255.255.0,destIp=10.11.18.7
2014-07-16 06:18:21,188 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: /bin/bash -c ip route delete 10.11.18.7
2014-07-16 06:18:21,191 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:21,191 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: /bin/bash -c ip route add 10.11.18.7 via 10.11.16.254
2014-07-16 06:18:21,193 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:21,193 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) addRouteToInternalIp: added route to internal ip=10.11.18.7 via 10.11.16.254
2014-07-16 06:18:21,193 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: mount
2014-07-16 06:18:21,196 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:21,197 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Executing: /usr/local/cloud/systemvm/config_ssl.sh -i 10.11.17.32 -h s-914-VM -k /tmp/prvkey2711851077634269814.tmp -p /tmp/pubcert2514594449977992293.tmp
2014-07-16 06:18:21,905 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Request:Seq 46-1933115394: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100011, [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.19.3","port":"443","intf":"eth2"}],"isAppendAIp":true,"wait":0}}] }
2014-07-16 06:18:21,905 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Processing command: com.cloud.agent.api.SecStorageFirewallCfgCommand
2014-07-16 06:18:21,905 DEBUG [utils.script.Script] (agentRequest-Handler-4:) Executing: /usr/local/cloud/systemvm/ipfirewall.sh true 10.11.19.3
2014-07-16 06:18:21,908 DEBUG [utils.script.Script] (agentRequest-Handler-4:) Execution is successful.
2014-07-16 06:18:21,908 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Seq 46-1933115394: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}] }
2014-07-16 06:18:22,336 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:22,336 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Stopping web server: apache2apache2: Could not reliably determine the server's fully qualified domain name, using 10.11.17.32 for ServerName
... waiting .
Starting web server: apache2apache2: Could not reliably determine the server's fully qualified domain name, using 10.11.17.32 for ServerName
.
2014-07-16 06:18:22,336 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Seq 46-1933115393: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"SecStorageSetupAnswer":{"_dir":"a35b9ab9-d061-3ed7-bbe7-82022eec78e4","result":true,"details":"success","wait":0}}] }
2014-07-16 06:18:22,491 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Request:Seq 46-1933115395: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100011, [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.19.3","port":"443","intf":"eth2"}],"isAppendAIp":false,"wait":0}}] }
2014-07-16 06:18:22,491 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Processing command: com.cloud.agent.api.SecStorageFirewallCfgCommand
2014-07-16 06:18:22,491 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Executing: /usr/local/cloud/systemvm/ipfirewall.sh false 10.11.19.3
2014-07-16 06:18:22,497 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:22,497 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Seq 46-1933115395: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}] }
2014-07-16 06:18:22,587 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Request:Seq 46-1933115396: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"SecStorageVMSetupCommand":{"allowedInternalSites":["10.11.18.0/24","10.11.19.0/24","10.11.16.0/24","10.11.17.0/24","10.11.24.0/24","192.168.1.0/24"],"copyUserName":"cloud","copyPassword":"nD7iafqpzcsxqme","wait":0}}] }
2014-07-16 06:18:22,587 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Processing command: com.cloud.agent.api.SecStorageVMSetupCommand
2014-07-16 06:18:22,587 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.18.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:22,592 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,592 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.18.0/24
2014-07-16 06:18:22,592 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route delete 10.11.18.0/24
2014-07-16 06:18:22,594 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,594 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route add 10.11.18.0/24 via 10.11.16.254
2014-07-16 06:18:22,596 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,596 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: added route to internal ip=10.11.18.0/24 via 10.11.16.254
2014-07-16 06:18:22,596 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.19.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:22,600 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,601 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.19.0/24
2014-07-16 06:18:22,601 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route delete 10.11.19.0/24
2014-07-16 06:18:22,603 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,603 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route add 10.11.19.0/24 via 10.11.16.254
2014-07-16 06:18:22,605 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,605 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: added route to internal ip=10.11.19.0/24 via 10.11.16.254
2014-07-16 06:18:22,605 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.16.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:22,609 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,609 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.16.0/24
2014-07-16 06:18:22,609 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: dest ip 10.11.16.0/24 is in the same subnet as eth1 ip 10.11.16.51
2014-07-16 06:18:22,609 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.17.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:22,613 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,613 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.17.0/24
2014-07-16 06:18:22,613 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route delete 10.11.17.0/24
2014-07-16 06:18:22,616 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,616 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route add 10.11.17.0/24 via 10.11.16.254
2014-07-16 06:18:22,618 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,618 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: added route to internal ip=10.11.17.0/24 via 10.11.16.254
2014-07-16 06:18:22,618 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.24.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:22,622 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,622 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.24.0/24
2014-07-16 06:18:22,622 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route delete 10.11.24.0/24
2014-07-16 06:18:22,624 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,624 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route add 10.11.24.0/24 via 10.11.16.254
2014-07-16 06:18:22,626 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,626 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: added route to internal ip=10.11.24.0/24 via 10.11.16.254
2014-07-16 06:18:22,626 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 192.168.1.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:22,630 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,630 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=192.168.1.0/24
2014-07-16 06:18:22,631 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route delete 192.168.1.0/24
2014-07-16 06:18:22,633 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,633 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route add 192.168.1.0/24 via 10.11.16.254
2014-07-16 06:18:22,635 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,635 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: added route to internal ip=192.168.1.0/24 via 10.11.16.254
2014-07-16 06:18:22,635 DEBUG [utils.script.Script] (agentRequest-Handler-3:) Executing: /usr/local/cloud/systemvm/config_auth.sh cloud nD7iafqpzcsxqme
2014-07-16 06:18:22,640 DEBUG [utils.script.Script] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:22,640 DEBUG [utils.script.Script] (agentRequest-Handler-3:) Adding password for user cloud
2014-07-16 06:18:22,641 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Seq 46-1933115396: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"Answer":{"result":true,"details":"","wait":0}}] }
2014-07-16 06:18:22,685 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Request:Seq 46-1933115397: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"storage.ListTemplateCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] }
2014-07-16 06:18:22,686 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Processing command: com.cloud.agent.api.storage.ListTemplateCommand
2014-07-16 06:18:22,686 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:) Executing: mount
2014-07-16 06:18:22,688 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:) Execution is successful.
2014-07-16 06:18:22,689 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/
2014-07-16 06:18:22,969 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Execution is successful.
2014-07-16 06:18:22,969 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) found 28 templates[/mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/4/e970d345-e30b-30c4-879c-88d63200a721.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/3/6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/218/d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/254/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/209/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/216/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/264/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/261/c90f1ba1-8610-3dbb-bb6e-2bcad6ff75be.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/217/3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/287/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/280/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/273/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/249/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/258/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/263/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/283/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/267/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/282/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/208/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/265/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/215/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/210/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/211/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/262/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/268/c036ba8f-7976-342b-a82a-2fb58d5724ac.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/252/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/212/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/221/221-2-3706787c-d305-39e0-9680-9f84359df07a.iso]
2014-07-16 06:18:22,971 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Added template name: centos55-x86_64, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/4/e970d345-e30b-30c4-879c-88d63200a721.qcow2
2014-07-16 06:18:22,974 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Added template name: routing-3, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/3/6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2
2014-07-16 06:18:22,976 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Added template name: 218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/218/d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2
2014-07-16 06:18:22,978 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/254
2014-07-16 06:18:22,984 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/209
2014-07-16 06:18:22,990 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/216
2014-07-16 06:18:22,995 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/264
2014-07-16 06:18:23,001 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/261
2014-07-16 06:18:23,006 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Added template name: 217-2-0245468c-5e25-3513-92a4-89b4a045751f, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/217/3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2
2014-07-16 06:18:23,008 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/287
2014-07-16 06:18:23,014 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/280
2014-07-16 06:18:23,019 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/273
2014-07-16 06:18:23,025 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/249
2014-07-16 06:18:23,030 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/258
2014-07-16 06:18:23,035 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/263
2014-07-16 06:18:23,041 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/283
2014-07-16 06:18:23,047 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/267
2014-07-16 06:18:23,052 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/282
2014-07-16 06:18:23,058 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/208
2014-07-16 06:18:23,063 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/265
2014-07-16 06:18:23,068 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/215
2014-07-16 06:18:23,074 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/210
2014-07-16 06:18:23,079 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/211
2014-07-16 06:18:23,085 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/262
2014-07-16 06:18:23,090 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/268
2014-07-16 06:18:23,096 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/252
2014-07-16 06:18:23,102 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/212
2014-07-16 06:18:23,107 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Added template name: 221-2-3706787c-d305-39e0-9680-9f84359df07a, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/221/221-2-3706787c-d305-39e0-9680-9f84359df07a.iso
2014-07-16 06:18:23,108 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Seq 46-1933115397: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"storage.ListTemplateAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{"221-2-3706787c-d305-39e0-9680-9f84359df07a":{"templateName":"221-2-3706787c-d305-39e0-9680-9f84359df07a","installPath":"template/tmpl/2/221//221-2-3706787c-d305-39e0-9680-9f84359df07a.iso","size":45762560,"physicalSize":45762560,"id":221,"isPublic":true,"isCorrupted":false},"217-2-0245468c-5e25-3513-92a4-89b4a045751f":{"templateName":"217-2-0245468c-5e25-3513-92a4-89b4a045751f","installPath":"template/tmpl/2/217//3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2","size":107374182400,"physicalSize":4641128448,"id":217,"isPublic":true,"isCorrupted":false},"routing-3":{"templateName":"routing-3","installPath":"template/tmpl/1/3//6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2","size":725811200,"physicalSize":725811200,"id":3,"isPublic":true,"isCorrupted":false},"centos55-x86_64":{"templateName":"centos55-x86_64","installPath":"template/tmpl/1/4//e970d345-e30b-30c4-879c-88d63200a721.qcow2","size":8589934592,"physicalSize":1769537536,"id":4,"isPublic":true,"isCorrupted":false},"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1":{"templateName":"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1","installPath":"template/tmpl/2/218//d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2","size":107374182400,"physicalSize":15100084224,"id":218,"isPublic":true,"isCorrupted":false}},"result":true,"details":"success","wait":0}}] }
2014-07-16 06:18:23,430 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Request:Seq 46-1933115398: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"storage.ListVolumeCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] }
2014-07-16 06:18:23,430 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Processing command: com.cloud.agent.api.storage.ListVolumeCommand
2014-07-16 06:18:23,431 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-4:) Executing: mount
2014-07-16 06:18:23,434 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-4:) Execution is successful.
2014-07-16 06:18:23,435 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-4:) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvolume.sh -r /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/volumes/
2014-07-16 06:18:23,444 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-4:) Execution is successful.
2014-07-16 06:18:23,445 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-4:) found 0 volumes[]
2014-07-16 06:18:23,445 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Seq 46-1933115398: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"storage.ListVolumeAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{},"result":true,"details":"success","wait":0}}] }
2014-07-16 06:18:23,465 DEBUG [utils.nio.NioConnection] (Agent-Selector:) Location 1: Socket Socket[addr=/10.11.24.4,port=8250,localport=50809] closed on read. Probably -1 returned: Connection closed with -1 on reading size.
2014-07-16 06:18:23,466 DEBUG [utils.nio.NioConnection] (Agent-Selector:) Closing socket Socket[addr=/10.11.24.4,port=8250,localport=50809]
2014-07-16 06:18:23,466 DEBUG [cloud.agent.Agent] (Agent-Handler-5:) Clearing watch list: 1
2014-07-16 06:18:25,947 INFO [cloud.agent.Agent] (Agent-Handler-4:) Connected to the server
2014-07-16 06:18:28,467 INFO [cloud.agent.Agent] (Agent-Handler-5:) Lost connection to the server. Dealing with the remaining commands...
2014-07-16 06:18:33,468 INFO [cloud.agent.Agent] (Agent-Handler-5:) Reconnecting...
2014-07-16 06:18:33,469 INFO [utils.nio.NioClient] (Agent-Selector:) Connecting to 10.11.24.4:8250
2014-07-16 06:18:33,555 INFO [utils.nio.NioClient] (Agent-Selector:) SSL: Handshake done
2014-07-16 06:18:33,556 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for private nic: 10.11.16.51 - 06:5e:88:00:00:01-255.255.255.0
2014-07-16 06:18:33,557 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for storage nic: 10.11.16.51 - 06:5e:88:00:00:01-255.255.255.0
2014-07-16 06:18:33,557 DEBUG [cloud.resource.ServerResourceBase] (Agent-Handler-1:) Parameters for pubic nic: 10.11.17.32 - 06:c4:0c:00:00:48-255.255.255.0
2014-07-16 06:18:33,557 DEBUG [storage.resource.NfsSecondaryStorageResource] (Agent-Handler-1:) Executing: /bin/bash -c ln -sf /mnt/SecStorage /var/www/html/copy
2014-07-16 06:18:33,560 DEBUG [storage.resource.NfsSecondaryStorageResource] (Agent-Handler-1:) Execution is successful.
2014-07-16 06:18:33,560 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Executing: hostname
2014-07-16 06:18:33,561 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Execution is successful.
2014-07-16 06:18:33,561 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Sending Startup: Seq 46-334: { Cmd , MgmtId: -1, via: 46, Ver: v1, Flags: 101, [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"2","pod":"2","guid":"s-914-VM-NfsSecondaryStorageResource","name":"s-914-VM","id":46,"version":"2.2.20130913151423","iqn":"NoIqn","publicIpAddress":"10.11.17.32","publicNetmask":"255.255.255.0","publicMacAddress":"06:c4:0c:00:00:48","privateIpAddress":"10.11.16.51","privateMacAddress":"06:5e:88:00:00:01","privateNetmask":"255.255.255.0","storageIpAddress":"10.11.16.51","storageNetmask":"255.255.255.0","storageMacAddress":"06:5e:88:00:00:01","resourceName":"NfsSecondaryStorageResource","wait":0}}] }
2014-07-16 06:18:33,561 DEBUG [cloud.agent.Agent] (Agent-Handler-1:) Startup task created
2014-07-16 06:18:33,762 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Received response: Seq 46-334: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100100, [{"StartupAnswer":{"hostId":46,"pingInterval":60,"result":true,"wait":0}}] }
2014-07-16 06:18:33,765 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Startup task cancelled
2014-07-16 06:18:33,765 INFO [cloud.agent.Agent] (Agent-Handler-2:) Proccess agent startup answer, agent id = 46
2014-07-16 06:18:33,765 INFO [cloud.agent.Agent] (Agent-Handler-2:) Set agent id 46
2014-07-16 06:18:33,765 DEBUG [cloud.agent.Agent] (Agent-Handler-2:) Adding a watch list
2014-07-16 06:18:33,765 INFO [cloud.agent.Agent] (Agent-Handler-2:) Startup Response Received: agent id = 46
2014-07-16 06:18:33,765 DEBUG [cloud.agent.Agent] (UgentTask-5:) Sending ping: Seq 46-335: { Cmd , MgmtId: -1, via: 46, Ver: v1, Flags: 11, [{"PingStorageCommand":{"changes":{},"hostType":"Storage","hostId":46,"wait":0}}] }
2014-07-16 06:18:33,766 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Request:Seq 46-1592393729: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 10100111, [{"SecStorageSetupCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","certs":{},"wait":0}}] }
2014-07-16 06:18:33,766 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Processing command: com.cloud.agent.api.SecStorageSetupCommand
2014-07-16 06:18:33,766 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.80, eth1mask=255.255.255.0,destIp=10.11.18.7
2014-07-16 06:18:33,766 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: /bin/bash -c ip route delete 10.11.18.7
2014-07-16 06:18:33,769 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:33,769 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: /bin/bash -c ip route add 10.11.18.7 via 10.11.16.254
2014-07-16 06:18:33,772 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:33,772 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) addRouteToInternalIp: added route to internal ip=10.11.18.7 via 10.11.16.254
2014-07-16 06:18:33,773 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: mount
2014-07-16 06:18:33,775 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:33,775 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Executing: /usr/local/cloud/systemvm/config_ssl.sh -i 10.11.17.32 -h s-914-VM -k /tmp/prvkey8460466906045843370.tmp -p /tmp/pubcert3535456960738248048.tmp
2014-07-16 06:18:33,808 DEBUG [cloud.agent.Agent] (Agent-Handler-4:) Received response: Seq 46-335: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100010, [{"PingAnswer":{"_command":{"hostType":"Storage","hostId":46,"wait":0},"result":true,"wait":0}}] }
2014-07-16 06:18:34,967 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:34,968 DEBUG [utils.script.Script] (agentRequest-Handler-5:) Stopping web server: apache2apache2: Could not reliably determine the server's fully qualified domain name, using 10.11.17.32 for ServerName
... waiting .
Starting web server: apache2apache2: Could not reliably determine the server's fully qualified domain name, using 10.11.17.32 for ServerName
.
2014-07-16 06:18:34,968 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Seq 46-1592393729: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"SecStorageSetupAnswer":{"_dir":"a35b9ab9-d061-3ed7-bbe7-82022eec78e4","result":true,"details":"success","wait":0}}] }
2014-07-16 06:18:35,153 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Request:Seq 46-1592393730: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100011, [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.19.3","port":"443","intf":"eth2"}],"isAppendAIp":false,"wait":0}}] }
2014-07-16 06:18:35,153 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Processing command: com.cloud.agent.api.SecStorageFirewallCfgCommand
2014-07-16 06:18:35,153 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Executing: /usr/local/cloud/systemvm/ipfirewall.sh false 10.11.19.3
2014-07-16 06:18:35,157 DEBUG [utils.script.Script] (agentRequest-Handler-2:) Execution is successful.
2014-07-16 06:18:35,158 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:) Seq 46-1592393730: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}] }
2014-07-16 06:18:35,229 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Request:Seq 46-1592393731: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"SecStorageVMSetupCommand":{"allowedInternalSites":["10.11.18.0/24","10.11.19.0/24","10.11.16.0/24","10.11.17.0/24","10.11.24.0/24","192.168.1.0/24"],"copyUserName":"cloud","copyPassword":"nD7iafqpzcsxqme","wait":0}}] }
2014-07-16 06:18:35,229 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Processing command: com.cloud.agent.api.SecStorageVMSetupCommand
2014-07-16 06:18:35,229 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.18.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:35,233 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,233 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.18.0/24
2014-07-16 06:18:35,234 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route delete 10.11.18.0/24
2014-07-16 06:18:35,236 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,236 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route add 10.11.18.0/24 via 10.11.16.254
2014-07-16 06:18:35,238 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,238 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: added route to internal ip=10.11.18.0/24 via 10.11.16.254
2014-07-16 06:18:35,238 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.19.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:35,242 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,243 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.19.0/24
2014-07-16 06:18:35,243 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route delete 10.11.19.0/24
2014-07-16 06:18:35,245 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,245 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route add 10.11.19.0/24 via 10.11.16.254
2014-07-16 06:18:35,247 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,247 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: added route to internal ip=10.11.19.0/24 via 10.11.16.254
2014-07-16 06:18:35,247 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.16.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:35,251 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,251 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.16.0/24
2014-07-16 06:18:35,251 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: dest ip 10.11.16.0/24 is in the same subnet as eth1 ip 10.11.16.51
2014-07-16 06:18:35,252 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.17.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:35,256 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,256 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.17.0/24
2014-07-16 06:18:35,256 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route delete 10.11.17.0/24
2014-07-16 06:18:35,258 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,258 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route add 10.11.17.0/24 via 10.11.16.254
2014-07-16 06:18:35,260 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,260 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: added route to internal ip=10.11.17.0/24 via 10.11.16.254
2014-07-16 06:18:35,260 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 10.11.24.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:35,265 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,265 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=10.11.24.0/24
2014-07-16 06:18:35,265 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route delete 10.11.24.0/24
2014-07-16 06:18:35,267 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,267 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route add 10.11.24.0/24 via 10.11.16.254
2014-07-16 06:18:35,269 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,270 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: added route to internal ip=10.11.24.0/24 via 10.11.16.254
2014-07-16 06:18:35,270 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c iptables -I OUTPUT -o eth1 -d 192.168.1.0/24 -p tcp -m state --state NEW -m tcp -j ACCEPT
2014-07-16 06:18:35,274 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,274 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: localgw=10.11.16.254, eth1ip=10.11.16.51, eth1mask=255.255.255.0,destIp=192.168.1.0/24
2014-07-16 06:18:35,274 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route delete 192.168.1.0/24
2014-07-16 06:18:35,276 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,276 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Executing: /bin/bash -c ip route add 192.168.1.0/24 via 10.11.16.254
2014-07-16 06:18:35,278 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,278 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:) addRouteToInternalIp: added route to internal ip=192.168.1.0/24 via 10.11.16.254
2014-07-16 06:18:35,278 DEBUG [utils.script.Script] (agentRequest-Handler-3:) Executing: /usr/local/cloud/systemvm/config_auth.sh cloud nD7iafqpzcsxqme
2014-07-16 06:18:35,285 DEBUG [utils.script.Script] (agentRequest-Handler-3:) Execution is successful.
2014-07-16 06:18:35,285 DEBUG [utils.script.Script] (agentRequest-Handler-3:) Adding password for user cloud
2014-07-16 06:18:35,285 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:) Seq 46-1592393731: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"Answer":{"result":true,"details":"","wait":0}}] }
2014-07-16 06:18:35,349 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Request:Seq 46-1592393732: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"storage.ListTemplateCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] }
2014-07-16 06:18:35,349 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Processing command: com.cloud.agent.api.storage.ListTemplateCommand
2014-07-16 06:18:35,350 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:) Executing: mount
2014-07-16 06:18:35,352 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-1:) Execution is successful.
2014-07-16 06:18:35,354 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/
2014-07-16 06:18:35,559 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Request:Seq 46-1592393733: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100011, [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.19.3","port":"443","intf":"eth2"}],"isAppendAIp":true,"wait":0}}] }
2014-07-16 06:18:35,559 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Processing command: com.cloud.agent.api.SecStorageFirewallCfgCommand
2014-07-16 06:18:35,559 DEBUG [utils.script.Script] (agentRequest-Handler-4:) Executing: /usr/local/cloud/systemvm/ipfirewall.sh true 10.11.19.3
2014-07-16 06:18:35,562 DEBUG [utils.script.Script] (agentRequest-Handler-4:) Execution is successful.
2014-07-16 06:18:35,563 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:) Seq 46-1592393733: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}] }
2014-07-16 06:18:35,660 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Execution is successful.
2014-07-16 06:18:35,660 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) found 28 templates[/mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/4/e970d345-e30b-30c4-879c-88d63200a721.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/3/6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/218/d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/254/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/209/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/216/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/264/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/261/c90f1ba1-8610-3dbb-bb6e-2bcad6ff75be.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/217/3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/287/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/280/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/273/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/249/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/258/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/263/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/283/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/267/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/282/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/208/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/265/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/215/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/210/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/211/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/262/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/268/c036ba8f-7976-342b-a82a-2fb58d5724ac.qcow2, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/252/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/212/, /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/221/221-2-3706787c-d305-39e0-9680-9f84359df07a.iso]
2014-07-16 06:18:35,663 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Added template name: centos55-x86_64, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/4/e970d345-e30b-30c4-879c-88d63200a721.qcow2
2014-07-16 06:18:35,666 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Added template name: routing-3, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/1/3/6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2
2014-07-16 06:18:35,669 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Added template name: 218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/218/d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2
2014-07-16 06:18:35,671 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/254
2014-07-16 06:18:35,678 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/209
2014-07-16 06:18:35,684 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/216
2014-07-16 06:18:35,690 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/264
2014-07-16 06:18:35,696 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/261
2014-07-16 06:18:35,702 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Added template name: 217-2-0245468c-5e25-3513-92a4-89b4a045751f, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/217/3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2
2014-07-16 06:18:35,704 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/287
2014-07-16 06:18:35,710 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/280
2014-07-16 06:18:35,716 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/273
2014-07-16 06:18:35,722 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/249
2014-07-16 06:18:35,728 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/258
2014-07-16 06:18:35,734 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/263
2014-07-16 06:18:35,741 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/283
2014-07-16 06:18:35,747 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/267
2014-07-16 06:18:35,753 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/282
2014-07-16 06:18:35,759 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/208
2014-07-16 06:18:35,765 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/265
2014-07-16 06:18:35,771 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/215
2014-07-16 06:18:35,777 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/210
2014-07-16 06:18:35,783 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/211
2014-07-16 06:18:35,789 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/262
2014-07-16 06:18:35,795 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/268
2014-07-16 06:18:35,801 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/252
2014-07-16 06:18:35,807 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Post download installation was not completed for /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/212
2014-07-16 06:18:35,813 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:) Added template name: 221-2-3706787c-d305-39e0-9680-9f84359df07a, path: /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/template/tmpl/2/221/221-2-3706787c-d305-39e0-9680-9f84359df07a.iso
2014-07-16 06:18:35,814 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:) Seq 46-1592393732: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"storage.ListTemplateAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{"221-2-3706787c-d305-39e0-9680-9f84359df07a":{"templateName":"221-2-3706787c-d305-39e0-9680-9f84359df07a","installPath":"template/tmpl/2/221//221-2-3706787c-d305-39e0-9680-9f84359df07a.iso","size":45762560,"physicalSize":45762560,"id":221,"isPublic":true,"isCorrupted":false},"217-2-0245468c-5e25-3513-92a4-89b4a045751f":{"templateName":"217-2-0245468c-5e25-3513-92a4-89b4a045751f","installPath":"template/tmpl/2/217//3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2","size":107374182400,"physicalSize":4641128448,"id":217,"isPublic":true,"isCorrupted":false},"routing-3":{"templateName":"routing-3","installPath":"template/tmpl/1/3//6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2","size":725811200,"physicalSize":725811200,"id":3,"isPublic":true,"isCorrupted":false},"centos55-x86_64":{"templateName":"centos55-x86_64","installPath":"template/tmpl/1/4//e970d345-e30b-30c4-879c-88d63200a721.qcow2","size":8589934592,"physicalSize":1769537536,"id":4,"isPublic":true,"isCorrupted":false},"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1":{"templateName":"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1","installPath":"template/tmpl/2/218//d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2","size":107374182400,"physicalSize":15100084224,"id":218,"isPublic":true,"isCorrupted":false}},"result":true,"details":"success","wait":0}}] }
2014-07-16 06:18:36,179 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Request:Seq 46-1592393734: { Cmd , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 100111, [{"storage.ListVolumeCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] }
2014-07-16 06:18:36,179 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Processing command: com.cloud.agent.api.storage.ListVolumeCommand
2014-07-16 06:18:36,180 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Executing: mount
2014-07-16 06:18:36,181 DEBUG [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:36,182 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-5:) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvolume.sh -r /mnt/SecStorage/a35b9ab9-d061-3ed7-bbe7-82022eec78e4/volumes/
2014-07-16 06:18:36,192 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-5:) Execution is successful.
2014-07-16 06:18:36,192 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-5:) found 0 volumes[]
2014-07-16 06:18:36,192 DEBUG [cloud.agent.Agent] (agentRequest-Handler-5:) Seq 46-1592393734: { Ans: , MgmtId: 73548179740843, via: 46, Ver: v1, Flags: 110, [{"storage.ListVolumeAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{},"result":true,"details":"success","wait":0}}] }
2014-07-16 06:18:36,215 DEBUG [utils.nio.NioConnection] (Agent-Selector:) Location 1: Socket Socket[addr=/10.11.24.4,port=8250,localport=50810] closed on read. Probably -1 returned: Connection closed with -1 on reading size.
2014-07-16 06:18:36,216 DEBUG [utils.nio.NioConnection] (Agent-Selector:) Closing socket Socket[addr=/10.11.24.4,port=8250,localport=50810]
2014-07-16 06:18:36,216 DEBUG [cloud.agent.Agent] (Agent-Handler-5:) Clearing watch list: 1
------------------ 原始邮件 ------------------
发件人: "Wei ZHOU";<us...@gmail.com>;
发送时间: 2014年7月15日(星期二) 晚上6:40
收件人: "users-cn"<us...@cloudstack.apache.org>;
主题: Re: 回复：RE: 注册模板后出现状态：Storage agent or storage VM disconnected
你看一下ssvm里面的log吧
/var/log/cloud/目录
2014-07-15 10:34 GMT+02:00 赵涛 <zh...@qq.com>:
> 在vm_instance表里 s-909-VM/ s-889-VM是Running ，但在
> host表里显示状态是Alerting，销毁重建后还是一样
> ------------------ 原始邮件 ------------------
> 发件人: "YUAN, GUI -HBIS";<yu...@hbis.com.cn>;
> 发送时间: 2014年7月15日(星期二) 下午2:58
> 收件人: "users-cn"<us...@cloudstack.apache.org>;
>
> 主题: RE: 注册模板后出现状态：Storage agent or storage VM disconnected
>
>
>
> 1.SSVM没有启动，可能是对应的AgentHost连不上。确认有可用的HOST
> 如：s-909-VM/ s-889-VM
>
> 2.如果SSVM正常，模板添加时，提示SSVM- response time out
> 则需要确认NFS服务器的NFS服务状态及防火墙设置
>
>
>
> -----Original Message-----
> From: 赵涛 [mailto:zhaotao_cn@qq.com]
> Sent: Tuesday, July 15, 2014 12:46 PM
> To: users-cn
> Subject: 注册模板后出现状态：Storage agent or storage VM disconnected
>
> 注册模板后出现状态：Storage agent or storage VM disconnected,删除SSVM后问题依旧，请大神指导
>
> 删除SSVM虚拟机后重建的SSVM虚拟机上报日志：
>
>
> 2014-07-15 10:41:46,679 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-7:null) create ClusteredAgentAttache for 41
> 2014-07-15 10:41:46,685 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Connect to listener:
> XcpServerDiscoverer$$EnhancerByCGLIB$$18f3e761
> 2014-07-15 10:41:46,685 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Connect to listener:
> VmwareManagerImpl$$EnhancerByCGLIB$$4ad0e44c
> 2014-07-15 10:41:46,685 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Connect to listener:
> ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$15bc7206
> 2014-07-15 10:41:46,685 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Connect to listener:
> SecurityGroupListener
> 2014-07-15 10:41:46,685 INFO [network.security.SecurityGroupListener]
> (AgentManager-Handler-7:null) Received a host startup notification
> 2014-07-15 10:41:46,687 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Connect to listener:
> StoragePoolMonitor
> 2014-07-15 10:41:46,687 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Connect to listener:
> SecondaryStorageListener
> 2014-07-15 10:41:46,687 INFO [storage.secondary.SecondaryStorageListener]
> (AgentManager-Handler-7:null) Received a host startup notification
> com.cloud.agent.api.StartupSecondaryStorageCommand
> 2014-07-15 10:41:46,714 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 41-766640129: Sending { Cmd , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 100111,
> [{"SecStorageSetupCommand":{"secUrl":"nfs://10.11.18.7/export/d2secondary","certs":{},"wait":0}}]
> }
> 2014-07-15 10:41:46,727 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-14:null) Ping from 41
> 2014-07-15 10:41:46,776 DEBUG [agent.transport.Request]
> (AgentManager-Handler-10:null) Seq 42-1808072705: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110,
> [{"SecStorageSetupAnswer":{"_dir":"a35b9ab9-d061-3ed7-bbe7-82022eec78e4","result":true,"details":"success","wait":0}}]
> }
> 2014-07-15 10:41:46,776 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 42-1808072705: Received: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, {
> SecStorageSetupAnswer } }
> 2014-07-15 10:41:46,777 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-10:null) Seq 42-1808072705: No more commands found
> 2014-07-15 10:41:46,777 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Details from executing class
> com.cloud.agent.api.SecStorageSetupCommand: success
> 2014-07-15 10:41:46,809 DEBUG
> [storage.secondary.SecondaryStorageManagerImpl]
> (AgentManager-Handler-2:null) Successfully programmed secondary storage
> nfs://10.11.18.7/export/d3secondary in secondary storage VM s-909-VM
> 2014-07-15 10:41:46,831 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 41-766640130: Sending { Cmd , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 100011,
> [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.17.1","port":"443","intf":"eth2"}],"isAppendAIp":true,"wait":0}}]
> }
> 2014-07-15 10:41:46,880 DEBUG [agent.transport.Request]
> (AgentManager-Handler-4:null) Seq 41-766640130: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 10,
> [{"Answer":{"result":true,"wait":0}}] }
> 2014-07-15 10:41:46,881 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 41-766640130: Received: { Ans: , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 10, { Answer } }
> 2014-07-15 10:41:46,881 DEBUG
> [storage.secondary.SecondaryStorageManagerImpl]
> (AgentManager-Handler-2:null) Successfully programmed firewall rules into
> SSVM s-889-VM
> 2014-07-15 10:41:46,891 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 42-1808072706: Sending { Cmd , MgmtId:
> 73548179740843, via: 42, Ver: v1, Flags: 100011,
> [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.19.8","port":"443","intf":"eth2"}],"isAppendAIp":false,"wait":0}}]
> }
> 2014-07-15 10:41:46,901 DEBUG [agent.transport.Request]
> (AgentManager-Handler-8:null) Seq 42-1808072706: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 10,
> [{"Answer":{"result":true,"wait":0}}] }
> 2014-07-15 10:41:46,902 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 42-1808072706: Received: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 10, { Answer } }
> 2014-07-15 10:41:46,902 DEBUG
> [storage.secondary.SecondaryStorageManagerImpl]
> (AgentManager-Handler-2:null) Successfully programmed firewall rules into
> s-909-VM
> 2014-07-15 10:41:46,929 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 42-1808072707: Sending { Cmd , MgmtId:
> 73548179740843, via: 42, Ver: v1, Flags: 100111,
> [{"SecStorageVMSetupCommand":{"allowedInternalSites":["10.11.18.0/24","
> 10.11.19.0/24","10.11.16.0/24","10.11.17.0/24","10.11.24.0/24","
> 192.168.1.0/24"],"copyUserName":"cloud","copyPassword":"nD7iafqpzcsxqme","wait":0}}]
> }
> 2014-07-15 10:41:47,083 DEBUG [agent.transport.Request]
> (AgentManager-Handler-11:null) Seq 42-1808072707: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110,
> [{"Answer":{"result":true,"details":"","wait":0}}] }
> 2014-07-15 10:41:47,083 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-11:null) Seq 42-1808072707: No more commands found
> 2014-07-15 10:41:47,083 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 42-1808072707: Received: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, { Answer } }
> 2014-07-15 10:41:47,083 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Details from executing class
> com.cloud.agent.api.SecStorageVMSetupCommand:
> 2014-07-15 10:41:47,083 DEBUG
> [storage.secondary.SecondaryStorageManagerImpl]
> (AgentManager-Handler-2:null) Successfully programmed http auth into
> s-909-VM
> 2014-07-15 10:41:47,083 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Connect to listener:
> NetworkManagerImpl$$EnhancerByCGLIB$$bf7c5122
> 2014-07-15 10:41:47,083 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Connect to listener:
> AgentMonitor$$EnhancerByCGLIB$$301cd382
> 2014-07-15 10:41:47,084 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Connect to listener: DownloadListener
> 2014-07-15 10:41:47,101 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 42-1808072708: Sending { Cmd , MgmtId:
> 73548179740843, via: 42, Ver: v1, Flags: 100111,
> [{"storage.ListTemplateCommand":{"secUrl":"nfs://
> 10.11.18.7/export/d3secondary","wait":0}}] }
> 2014-07-15 10:41:47,629 DEBUG [agent.transport.Request]
> (AgentManager-Handler-3:null) Seq 42-1808072708: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110,
> [{"storage.ListTemplateAnswer":{"secUrl":"nfs://
> 10.11.18.7/export/d3secondary","templateInfos":{"221-2-3706787c-d305-39e0-9680-9f84359df07a":{"templateName":"221-2-3706787c-d305-39e0-9680-9f84359df07a","installPath":"template/tmpl/2/221//221-2-3706787c-d305-39e0-9680-9f84359df07a.iso","size":45762560,"physicalSize":45762560,"id":221,"isPublic":true,"isCorrupted":false},"275-2-bb1fbd41-a14c-3f06-ba59-f815a250b45c":{"templateName":"275-2-bb1fbd41-a14c-3f06-ba59-f815a250b45c","installPath":"template/tmpl/2/275//90034c67-a7d5-33cb-be93-b3d0bd02de9d.qcow2","size":292001153024,"physicalSize":17294753792,"id":275,"isPublic":true,"isCorrupted":false},"217-2-0245468c-5e25-3513-92a4-89b4a045751f":{"templateName":"217-2-0245468c-5e25-3513-92a4-89b4a045751f","installPath":"template/tmpl/2/217//3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2","size":107374182400,"physicalSize":4641128448,"id":217,"isPublic":true,"isCorrupted":false},"routing-3":{"templateName":"routing-3","installPath":"template/tmpl/1/3//6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2","size":725811200,"physicalSize":725811200,"id":3,"isPublic":true,"isCorrupted":false},"centos55-x86_64":{"templateName":"centos55-x86_64","installPath":"template/tmpl/1/4//e970d345-e30b-30c4-879c-88d63200a721.qcow2","size":8589934592,"physicalSize":1769537536,"id":4,"isPublic":true,"isCorrupted":false},"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1":{"templateName":"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1","installPath":"template/tmpl/2/218//d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2","size":107374182400,"physicalSize":15100084224,"id":218,"isPublic":true,"isCorrupted":false}},"result":true,"details":"success","wait":0}}]
> }
> 2014-07-15 10:41:47,630 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-3:null) Seq 42-1808072708: No more commands found
> 2014-07-15 10:41:47,630 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 42-1808072708: Received: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, { ListTemplateAnswer
> } }
> 2014-07-15 10:41:47,630 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Details from executing class
> com.cloud.agent.api.storage.ListTemplateCommand: success
> 2014-07-15 10:41:47,653 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-2:null) Template Sync found routing-3 already in the
> template host table
> 2014-07-15 10:41:47,705 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-2:null) Template Sync found centos55-x86_64 already
> in the template host table
> 2014-07-15 10:41:47,771 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-2:null) Template Sync did not find
> centos56-x86_64-xen ready on server 11, will request download to
> start/resume shortly
> 2014-07-15 10:41:47,775 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-2:null) Template Sync did not find centos53-x64 ready
> on server 11, will request download to start/resume shortly
> 2014-07-15 10:41:47,785 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-2:null) Template Sync found
> 217-2-0245468c-5e25-3513-92a4-89b4a045751f already in the template host
> table
> 2014-07-15 10:41:47,837 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-2:null) Template Sync found
> 218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1 already in the template host
> table
> 2014-07-15 10:41:47,885 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-2:null) Template Sync found
> 221-2-3706787c-d305-39e0-9680-9f84359df07a already in the template host
> table
> 2014-07-15 10:41:47,913 DEBUG [agent.transport.Request]
> (AgentManager-Handler-6:null) Seq 41-766640129: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110,
> [{"SecStorageSetupAnswer":{"_dir":"a5fd862f-5d3e-3c41-b912-3918446c341a","result":true,"details":"success","wait":0}}]
> }
> 2014-07-15 10:41:47,913 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-6:null) Seq 41-766640129: No more commands found
> 2014-07-15 10:41:47,913 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 41-766640129: Received: { Ans: , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 110, { SecStorageSetupAnswer } }
> 2014-07-15 10:41:47,913 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Details from executing class
> com.cloud.agent.api.SecStorageSetupCommand: success
> 2014-07-15 10:41:47,933 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-2:null) Template Sync found
> 275-2-bb1fbd41-a14c-3f06-ba59-f815a250b45c already in the template host
> table
> 2014-07-15 10:41:47,946 DEBUG
> [storage.secondary.SecondaryStorageManagerImpl]
> (AgentManager-Handler-7:null) Successfully programmed secondary storage
> nfs://10.11.18.7/export/d2secondary in secondary storage VM s-889-VM
> 2014-07-15 10:41:47,968 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 42-1808072709: Sending { Cmd , MgmtId:
> 73548179740843, via: 42, Ver: v1, Flags: 100011,
> [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.19.8","port":"443","intf":"eth2"}],"isAppendAIp":true,"wait":0}}]
> }
> 2014-07-15 10:41:47,976 DEBUG [agent.transport.Request]
> (AgentManager-Handler-13:null) Seq 42-1808072709: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 10,
> [{"Answer":{"result":true,"wait":0}}] }
> 2014-07-15 10:41:47,977 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 42-1808072709: Received: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 10, { Answer } }
> 2014-07-15 10:41:47,977 DEBUG
> [storage.secondary.SecondaryStorageManagerImpl]
> (AgentManager-Handler-7:null) Successfully programmed firewall rules into
> SSVM s-909-VM
> 2014-07-15 10:41:47,985 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 41-766640131: Sending { Cmd , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 100011,
> [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.17.1","port":"443","intf":"eth2"}],"isAppendAIp":false,"wait":0}}]
> }
> 2014-07-15 10:41:48,016 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 42-1808072710: Sending { Cmd , MgmtId:
> 73548179740843, via: 42, Ver: v1, Flags: 100111,
> [{"storage.ListVolumeCommand":{"secUrl":"nfs://
> 10.11.18.7/export/d3secondary","wait":0}}] }
> 2014-07-15 10:41:48,033 DEBUG [agent.transport.Request]
> (AgentManager-Handler-5:null) Seq 41-766640131: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 10,
> [{"Answer":{"result":true,"wait":0}}] }
> 2014-07-15 10:41:48,033 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 41-766640131: Received: { Ans: , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 10, { Answer } }
> 2014-07-15 10:41:48,033 DEBUG
> [storage.secondary.SecondaryStorageManagerImpl]
> (AgentManager-Handler-7:null) Successfully programmed firewall rules into
> s-889-VM
> 2014-07-15 10:41:48,059 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 41-766640132: Sending { Cmd , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 100111,
> [{"SecStorageVMSetupCommand":{"allowedInternalSites":["10.11.18.0/24","
> 10.11.19.0/24","10.11.16.0/24","10.11.17.0/24","10.11.24.0/24","
> 192.168.1.0/24"],"copyUserName":"cloud","copyPassword":"nD7iafqpzcsxqme","wait":0}}]
> }
> 2014-07-15 10:41:48,077 DEBUG [agent.transport.Request]
> (AgentManager-Handler-12:null) Seq 42-1808072710: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110,
> [{"storage.ListVolumeAnswer":{"secUrl":"nfs://
> 10.11.18.7/export/d3secondary","templateInfos":{},"result":true,"details":"success","wait":0}}]
> }
> 2014-07-15 10:41:48,077 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-12:null) Seq 42-1808072710: No more commands found
> 2014-07-15 10:41:48,077 DEBUG [agent.transport.Request]
> (AgentManager-Handler-2:null) Seq 42-1808072710: Received: { Ans: ,
> MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, { ListVolumeAnswer } }
> 2014-07-15 10:41:48,077 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Details from executing class
> com.cloud.agent.api.storage.ListVolumeCommand: success
> 2014-07-15 10:41:48,085 ERROR [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Monitor DownloadListener says there is an
> error in the connect process for 42 due to null
> java.lang.NullPointerException
> at
> com.cloud.storage.download.DownloadMonitorImpl.handleVolumeSync(DownloadMonitorImpl.java:704)
> at
> com.cloud.storage.download.DownloadMonitorImpl.handleSync(DownloadMonitorImpl.java:630)
> at
> com.cloud.storage.download.DownloadListener.processConnect(DownloadListener.java:385)
> at
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:605)
> at
> com.cloud.agent.manager.AgentManagerImpl.handleConnectedAgent(AgentManagerImpl.java:1157)
> at
> com.cloud.agent.manager.AgentManagerImpl.access$100(AgentManagerImpl.java:142)
> at
> com.cloud.agent.manager.AgentManagerImpl$AgentHandler.processRequest(AgentManagerImpl.java:1235)
> at
> com.cloud.agent.manager.AgentManagerImpl$AgentHandler.doTask(AgentManagerImpl.java:1374)
> at
> com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:618)
> at com.cloud.utils.nio.Task.run(Task.java:83)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2014-07-15 10:41:48,089 INFO [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Host 42 is disconnecting with event
> AgentDisconnected
> 2014-07-15 10:41:48,093 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) The next status of agent 42is Alert, current
> status is Connecting
> 2014-07-15 10:41:48,093 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Deregistering link for 42 with state Alert
> 2014-07-15 10:41:48,093 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Remove Agent : 42
> 2014-07-15 10:41:48,093 DEBUG [agent.manager.ConnectedAgentAttache]
> (AgentManager-Handler-2:null) Processing Disconnect.
> 2014-07-15 10:41:48,094 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer$$EnhancerByCGLIB$$18f3e761
> 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.hypervisor.vmware.manager.VmwareManagerImpl$$EnhancerByCGLIB$$4ad0e44c
> 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.vm.ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$15bc7206
> 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.network.security.SecurityGroupListener
> 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.storage.listener.StoragePoolMonitor
> 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.storage.secondary.SecondaryStorageListener
> 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.network.NetworkManagerImpl$$EnhancerByCGLIB$$bf7c5122
> 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.agent.manager.AgentMonitor$$EnhancerByCGLIB$$301cd382
> 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.storage.download.DownloadListener
> 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.storage.upload.UploadListener
> 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.storage.LocalStoragePoolListener$$EnhancerByCGLIB$$c00af9b6
> 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.capacity.StorageCapacityListener
> 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.capacity.ComputeCapacityListener
> 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.consoleproxy.ConsoleProxyListener
> 2014-07-15 10:41:48,101 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.network.SshKeysDistriMonitor
> 2014-07-15 10:41:48,101 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl$$EnhancerByCGLIB$$c27578aa
> 2014-07-15 10:41:48,101 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Sending Disconnect to listener:
> com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
> 2014-07-15 10:41:48,101 DEBUG [cloud.network.NetworkUsageManagerImpl]
> (AgentManager-Handler-2:null) Disconnected called on 42 with status Alert
> 2014-07-15 10:41:48,101 DEBUG [cloud.host.Status]
> (AgentManager-Handler-2:null) Transition:[Resource state = Enabled, Agent
> event = AgentDisconnected, Host id = 42, name = s-909-VM]
> 2014-07-15 10:41:48,163 DEBUG [cloud.host.Status]
> (AgentManager-Handler-2:null) Agent status update: [id = 42; name =
> s-909-VM; old status = Connecting; event = AgentDisconnected; new status =
> Alert; old update count = 209; new update count = 210]
> 2014-07-15 10:41:48,163 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-2:null) Notifying other nodes of to disconnect
> 2014-07-15 10:41:48,178 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Can not send command
> com.cloud.agent.api.ReadyCommand due to Host 42 is not up
> 2014-07-15 10:41:48,178 WARN [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-2:null) Unable to create attache for agent: Seq
> 42-208: { Cmd , MgmtId: -1, via: 42, Ver: v1, Flags: 101,
> [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"2","pod":"2","guid":"s-909-VM-NfsSecondaryStorageResource","name":"s-909-VM","id":42,"version":"2.2.20130913151423","iqn":"NoIqn","publicIpAddress":"10.11.17.1","publicNetmask":"255.255.255.0","publicMacAddress":"06:ed:f0:00:00:29","privateIpAddress":"10.11.16.70","privateMacAddress":"06:5c:62:00:00:14","privateNetmask":"255.255.255.0","storageIpAddress":"10.11.16.70","storageNetmask":"255.255.255.0","storageMacAddress":"06:5c:62:00:00:14","resourceName":"NfsSecondaryStorageResource","wait":0}}]
> }
> 2014-07-15 10:41:48,208 DEBUG [agent.transport.Request]
> (AgentManager-Handler-9:null) Seq 41-766640132: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110,
> [{"Answer":{"result":true,"details":"","wait":0}}] }
> 2014-07-15 10:41:48,208 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-9:null) Seq 41-766640132: No more commands found
> 2014-07-15 10:41:48,208 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 41-766640132: Received: { Ans: , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 110, { Answer } }
> 2014-07-15 10:41:48,208 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Details from executing class
> com.cloud.agent.api.SecStorageVMSetupCommand:
> 2014-07-15 10:41:48,208 DEBUG
> [storage.secondary.SecondaryStorageManagerImpl]
> (AgentManager-Handler-7:null) Successfully programmed http auth into
> s-889-VM
> 2014-07-15 10:41:48,209 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Connect to listener:
> NetworkManagerImpl$$EnhancerByCGLIB$$bf7c5122
> 2014-07-15 10:41:48,209 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Connect to listener:
> AgentMonitor$$EnhancerByCGLIB$$301cd382
> 2014-07-15 10:41:48,209 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Connect to listener: DownloadListener
> 2014-07-15 10:41:48,227 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 41-766640133: Sending { Cmd , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 100111,
> [{"storage.ListTemplateCommand":{"secUrl":"nfs://
> 10.11.18.7/export/d2secondary","wait":0}}] }
> 2014-07-15 10:41:48,498 DEBUG [agent.transport.Request]
> (AgentManager-Handler-1:null) Seq 41-766640133: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110,
> [{"storage.ListTemplateAnswer":{"secUrl":"nfs://
> 10.11.18.7/export/d2secondary","templateInfos":{"221-2-3706787c-d305-39e0-9680-9f84359df07a":{"templateName":"221-2-3706787c-d305-39e0-9680-9f84359df07a","installPath":"template/tmpl/2/221//221-2-3706787c-d305-39e0-9680-9f84359df07a.iso","size":45762560,"physicalSize":45762560,"id":221,"isPublic":true,"isCorrupted":false},"206-2-33e4a593-fa36-3fb6-b68e-a587a8b1af80":{"templateName":"206-2-33e4a593-fa36-3fb6-b68e-a587a8b1af80","installPath":"template/tmpl/2/206//206-2-33e4a593-fa36-3fb6-b68e-a587a8b1af80.iso","size":3548971164,"physicalSize":3548971164,"id":206,"isPublic":true,"isCorrupted":false},"a1e5cdd4-b892-45a0-aeb4-a337cc96b442":{"templateName":"a1e5cdd4-b892-45a0-aeb4-a337cc96b442","installPath":"template/tmpl/2/209//a1e5cdd4-b892-45a0-aeb4-a337cc96b442.qcow2","size":107374182400,"physicalSize":15100084224,"id":1,"isPublic":true,"isCorrupted":false},"routing-3":{"templateName":"routing-3","installPath":"template/tmpl/1/3//3e38f707-5006-4d73-92a5-5be21b6f178f.qcow2","size":725811200,"physicalSize":725811200,"id":3,"isPublic":true,"isCorrupted":false},"5deebbfd-e603-4d09-912d-de5848d7118e":{"templateName":"5deebbfd-e603-4d09-912d-de5848d7118e","installPath":"template/tmpl/2/208//5deebbfd-e603-4d09-912d-de5848d7118e.qcow2","size":107374182400,"physicalSize":4641128448,"id":1,"isPublic":true,"isCorrupted":false},"205-2-5ffdab64-da2e-3afc-b166-061397e0de7c":{"templateName":"205-2-5ffdab64-da2e-3afc-b166-061397e0de7c","installPath":"template/tmpl/2/205//205-2-5ffdab64-da2e-3afc-b166-061397e0de7c.iso","size":4239566848,"physicalSize":4239566848,"id":205,"isPublic":true,"isCorrupted":false},"centos55-x86_64":{"templateName":"centos55-x86_64","installPath":"template/tmpl/1/4//20fb26b3-a01a-39ab-a39b-3b459dc862ed.qcow2","size":8589934592,"physicalSize":1769537536,"id":4,"isPublic":true,"isCorrupted":false}},"result":true,"details":"success","wait":0}}]
> }
> 2014-07-15 10:41:48,498 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-1:null) Seq 41-766640133: No more commands found
> 2014-07-15 10:41:48,498 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 41-766640133: Received: { Ans: , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 110, { ListTemplateAnswer } }
> 2014-07-15 10:41:48,498 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Details from executing class
> com.cloud.agent.api.storage.ListTemplateCommand: success
> 2014-07-15 10:41:48,521 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-7:null) Template Sync found routing-3 already in the
> template host table
> 2014-07-15 10:41:48,567 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-7:null) Template Sync found centos55-x86_64 already
> in the template host table
> 2014-07-15 10:41:48,609 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-7:null) Template Sync did not find
> centos56-x86_64-xen ready on server 2, will request download to
> start/resume shortly
> 2014-07-15 10:41:48,613 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-7:null) Template Sync did not find centos53-x64 ready
> on server 2, will request download to start/resume shortly
> 2014-07-15 10:41:48,625 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-7:null) Template Sync found
> 205-2-5ffdab64-da2e-3afc-b166-061397e0de7c already in the template host
> table
> 2014-07-15 10:41:48,687 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-7:null) Template Sync found
> 206-2-33e4a593-fa36-3fb6-b68e-a587a8b1af80 already in the template host
> table
> 2014-07-15 10:41:48,747 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-7:null) Template Sync found
> 5deebbfd-e603-4d09-912d-de5848d7118e already in the template host table
> 2014-07-15 10:41:48,813 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-7:null) Template Sync found
> a1e5cdd4-b892-45a0-aeb4-a337cc96b442 already in the template host table
> 2014-07-15 10:41:48,860 INFO [storage.download.DownloadMonitorImpl]
> (AgentManager-Handler-7:null) Template Sync found
> 221-2-3706787c-d305-39e0-9680-9f84359df07a already in the template host
> table
> 2014-07-15 10:41:48,922 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 41-766640134: Sending { Cmd , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 100111,
> [{"storage.ListVolumeCommand":{"secUrl":"nfs://
> 10.11.18.7/export/d2secondary","wait":0}}] }
> 2014-07-15 10:41:48,939 DEBUG [agent.transport.Request]
> (AgentManager-Handler-15:null) Seq 41-766640134: Processing: { Ans: ,
> MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110,
> [{"storage.ListVolumeAnswer":{"secUrl":"nfs://
> 10.11.18.7/export/d2secondary","templateInfos":{},"result":true,"details":"success","wait":0}}]
> }
> 2014-07-15 10:41:48,939 DEBUG [agent.manager.AgentAttache]
> (AgentManager-Handler-15:null) Seq 41-766640134: No more commands found
> 2014-07-15 10:41:48,939 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 41-766640134: Received: { Ans: , MgmtId:
> 73548179740843, via: 41, Ver: v1, Flags: 110, { ListVolumeAnswer } }
> 2014-07-15 10:41:48,940 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Details from executing class
> com.cloud.agent.api.storage.ListVolumeCommand: success
> 2014-07-15 10:41:48,948 ERROR [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Monitor DownloadListener says there is an
> error in the connect process for 41 due to null
> java.lang.NullPointerException
> at
> com.cloud.storage.download.DownloadMonitorImpl.handleVolumeSync(DownloadMonitorImpl.java:704)
> at
> com.cloud.storage.download.DownloadMonitorImpl.handleSync(DownloadMonitorImpl.java:630)
> at
> com.cloud.storage.download.DownloadListener.processConnect(DownloadListener.java:385)
> at
> com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:605)
> at
> com.cloud.agent.manager.AgentManagerImpl.handleConnectedAgent(AgentManagerImpl.java:1157)
> at
> com.cloud.agent.manager.AgentManagerImpl.access$100(AgentManagerImpl.java:142)
> at
> com.cloud.agent.manager.AgentManagerImpl$AgentHandler.processRequest(AgentManagerImpl.java:1235)
> at
> com.cloud.agent.manager.AgentManagerImpl$AgentHandler.doTask(AgentManagerImpl.java:1374)
> at
> com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:618)
> at com.cloud.utils.nio.Task.run(Task.java:83)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2014-07-15 10:41:48,951 INFO [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Host 41 is disconnecting with event
> AgentDisconnected
> 2014-07-15 10:41:48,956 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) The next status of agent 41is Alert, current
> status is Connecting
> 2014-07-15 10:41:48,956 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Deregistering link for 41 with state Alert
> 2014-07-15 10:41:48,956 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Remove Agent : 41
> 2014-07-15 10:41:48,956 DEBUG [agent.manager.ConnectedAgentAttache]
> (AgentManager-Handler-7:null) Processing Disconnect.
> 2014-07-15 10:41:48,957 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer$$EnhancerByCGLIB$$18f3e761
> 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.hypervisor.vmware.manager.VmwareManagerImpl$$EnhancerByCGLIB$$4ad0e44c
> 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.vm.ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$15bc7206
> 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.network.security.SecurityGroupListener
> 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.storage.listener.StoragePoolMonitor
> 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.storage.secondary.SecondaryStorageListener
> 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.network.NetworkManagerImpl$$EnhancerByCGLIB$$bf7c5122
> 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.agent.manager.AgentMonitor$$EnhancerByCGLIB$$301cd382
> 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.storage.download.DownloadListener
> 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.storage.upload.UploadListener
> 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.storage.LocalStoragePoolListener$$EnhancerByCGLIB$$c00af9b6
> 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.capacity.StorageCapacityListener
> 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.capacity.ComputeCapacityListener
> 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.consoleproxy.ConsoleProxyListener
> 2014-07-15 10:41:48,963 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.network.SshKeysDistriMonitor
> 2014-07-15 10:41:48,963 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl$$EnhancerByCGLIB$$c27578aa
> 2014-07-15 10:41:48,964 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Sending Disconnect to listener:
> com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
> 2014-07-15 10:41:48,964 DEBUG [cloud.network.NetworkUsageManagerImpl]
> (AgentManager-Handler-7:null) Disconnected called on 41 with status Alert
> 2014-07-15 10:41:48,964 DEBUG [cloud.host.Status]
> (AgentManager-Handler-7:null) Transition:[Resource state = Enabled, Agent
> event = AgentDisconnected, Host id = 41, name = s-889-VM]
> 2014-07-15 10:41:49,006 DEBUG [cloud.host.Status]
> (AgentManager-Handler-7:null) Agent status update: [id = 41; name =
> s-889-VM; old status = Connecting; event = AgentDisconnected; new status =
> Alert; old update count = 391; new update count = 392]
> 2014-07-15 10:41:49,006 DEBUG [agent.manager.ClusteredAgentManagerImpl]
> (AgentManager-Handler-7:null) Notifying other nodes of to disconnect
> 2014-07-15 10:41:49,014 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Can not send command
> com.cloud.agent.api.ReadyCommand due to Host 41 is not up
> 2014-07-15 10:41:49,015 WARN [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-7:null) Unable to create attache for agent: Seq
> 41-390: { Cmd , MgmtId: -1, via: 41, Ver: v1, Flags: 101,
> [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"1","pod":"1","guid":"s-889-VM-NfsSecondaryStorageResource","name":"s-889-VM","id":41,"version":"2.2.20130913151423","iqn":"NoIqn","publicIpAddress":"10.11.19.8","publicNetmask":"255.255.255.0","publicMacAddress":"06:fb:ee:00:00:1c","privateIpAddress":"10.11.18.67","privateMacAddress":"06:89:b0:00:00:11","privateNetmask":"255.255.255.0","storageIpAddress":"10.11.18.67","storageNetmask":"255.255.255.0","storageMacAddress":"06:89:b0:00:00:11","resourceName":"NfsSecondaryStorageResource","wait":0}}]
> }
> 2014-07-15 10:41:50,555 DEBUG
> [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Zone 1
> is ready to launch secondary storage VM
> 2014-07-15 10:41:50,609 DEBUG
> [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Zone 2
> is ready to launch secondary storage VM
> 2014-07-15 10:41:50,851 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl]
> (consoleproxy-1:null) Zone 1 is ready to launch console proxy
> 2014-07-15 10:41:50,868 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl]
> (consoleproxy-1:null) Zone 2 is ready to launch console proxy
> 2014-07-15 10:41:51,848 DEBUG [cloud.server.StatsCollector]
> (StatsCollector-1:null) StorageCollector is running...
> 2014-07-15 10:41:51,864 DEBUG [cloud.server.StatsCollector]
> (StatsCollector-1:null) There is no secondary storage VM for secondary
> storage host nfs://10.11.18.7/export/d2secondary
> 2014-07-15 10:41:51,869 DEBUG [cloud.server.StatsCollector]
> (StatsCollector-1:null) There is no secondary storage VM for secondary
> storage host nfs://10.11.18.7/export/d3secondary
> 2014-07-15 10:41:51,928 DEBUG
> [network.router.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:null) Found 0 routers.
> 2014-07-15 10:41:51,991 DEBUG [agent.transport.Request]
> (StatsCollector-1:null) Seq 7-513215195: Received: { Ans: , MgmtId:
> 73548179740843, via: 7, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> 2014-07-15 10:41:52,059 DEBUG [agent.transport.Request]
> (StatsCollector-1:null) Seq 9-1410599594: Received: { Ans: , MgmtId:
> 73548179740843, via: 9, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> 2014-07-15 10:41:52,128 DEBUG [agent.transport.Request]
> (StatsCollector-1:null) Seq 14-1320747056: Received: { Ans: , MgmtId:
> 73548179740843, via: 14, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> 2014-07-15 10:41:52,150 DEBUG [agent.transport.Request]
> (StatsCollector-1:null) Seq 16-940116823: Received: { Ans: , MgmtId:
> 73548179740843, via: 16, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> 2014-07-15 10:41:52,231 DEBUG [agent.transport.Request]
> (StatsCollector-1:null) Seq 14-1320747057: Received: { Ans: , MgmtId:
> 73548179740843, via: 14, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> 2014-07-15 10:41:52,312 DEBUG [agent.transport.Request]
> (StatsCollector-1:null) Seq 7-513215196: Received: { Ans: , MgmtId:
> 73548179740843, via: 7, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
> 2014-07-15 10:41:53,351 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-6:null) Ping from 23
> 2014-07-15 10:41:54,315 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-13:null) Ping from 8
> 2014-07-15 10:41:54,532 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-5:null) Ping from 40
>
> Disclaimer:
> The contents of this e-mail, and its attachments, if any, are confidential
> and may be protected
> by law against any unauthorized use. If you have received this e-mail by
> mistake or have
> reason to believe that you are not the intended recipient, please notify
> the sender by reply
> e-mail as soon as possible and delete it from your computer system
> immediately thereafter.
> If you are not the intended recipient, you must not copy this e-mail or
> attachment or disclose
> the contents to any other person. While we have made every effort to keep
> our network virus free,
> we take no responsibility for any computer virus which might be
> transferred by way of this e-mail.
>