You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Wei ZHOU <us...@gmail.com> on 2013/09/18 07:30:28 UTC
Re: [jira] [Commented] (CLOUDSTACK-4690) KVM Router - to many
ethernet devices created
Did you build the source code in Windows?
2013/9/18, Paul Edwards (JIRA) <ji...@apache.org>:
>
> [
> https://issues.apache.org/jira/browse/CLOUDSTACK-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770450#comment-13770450
> ]
>
> Paul Edwards commented on CLOUDSTACK-4690:
> ------------------------------------------
>
> As I said above, there was no activity in the agent log. I was tailing it at
> the time. Below I have pasted the entire log from the host where the vm was
> running, as you can see there was no action at the period expected. Now it
> maybe that I need to turn some logging on somewhere. If so please tell me
> where.
>
> Thanks
> P.
>
> 2013-09-17 06:59:14,628 INFO [cloud.agent.AgentShell] (main:null) Agent
> started
> 2013-09-17 06:59:14,630 INFO [cloud.agent.AgentShell] (main:null)
> Implementation Version is 4.1.1
> 2013-09-17 06:59:14,632 INFO [cloud.agent.AgentShell] (main:null)
> agent.properties found at /etc/cloudstack/agent/agent.properties
> 2013-09-17 06:59:14,634 INFO [cloud.agent.AgentShell] (main:null)
> Defaulting to using properties file for storage
> 2013-09-17 06:59:14,635 INFO [cloud.agent.AgentShell] (main:null)
> Defaulting to the constant time backoff algorithm
> 2013-09-17 06:59:14,704 INFO [cloud.agent.Agent] (main:null) id is 17
> 2013-09-17 06:59:14,724 INFO
> [resource.virtualnetwork.VirtualRoutingResource] (main:null)
> VirtualRoutingResource _scriptDir to use: scripts/network/domr/kvm
> 2013-09-17 06:59:15,313 INFO [kvm.resource.LibvirtComputingResource]
> (main:null) No libvirt.vif.driver specififed. Defaults to BridgeVifDriver.
> 2013-09-17 06:59:15,432 INFO [cloud.agent.Agent] (main:null) Agent [id = 17
> : type = LibvirtComputingResource : zone = 1 : pod = 1 : workers = 5 : host
> = xxx.xxx.245.48 : port = 8250
> 2013-09-17 06:59:15,450 INFO [utils.nio.NioClient] (Agent-Selector:null)
> Connecting to xxx.xxx.245.48:8250
> 2013-09-17 06:59:18,456 ERROR [utils.nio.NioConnection]
> (Agent-Selector:null) Unable to initialize the threads.
> java.net.NoRouteToHostException: No route to host
> at sun.nio.ch.Net.connect(Native Method)
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:530)
> at com.cloud.utils.nio.NioClient.init(NioClient.java:64)
> at com.cloud.utils.nio.NioConnection.run(NioConnection.java:108)
> at java.lang.Thread.run(Thread.java:679)
> 2013-09-17 06:59:23,475 INFO [utils.nio.NioClient] (Agent-Selector:null)
> Connecting to xxx.xxx.245.48:8250
> 2013-09-17 06:59:26,476 ERROR [utils.nio.NioConnection]
> (Agent-Selector:null) Unable to initialize the threads.
> java.net.NoRouteToHostException: No route to host
> at sun.nio.ch.Net.connect(Native Method)
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:530)
> at com.cloud.utils.nio.NioClient.init(NioClient.java:64)
> at com.cloud.utils.nio.NioConnection.run(NioConnection.java:108)
> at java.lang.Thread.run(Thread.java:679)
> 2013-09-17 06:59:31,482 INFO [utils.nio.NioClient] (Agent-Selector:null)
> Connecting to xxx.xxx.245.48:8250
> 2013-09-17 06:59:31,656 INFO [utils.nio.NioClient] (Agent-Selector:null)
> SSL: Handshake done
> 2013-09-17 06:59:31,911 INFO [cloud.serializer.GsonHelper]
> (Agent-Handler-1:null) Default Builder inited.
> 2013-09-17 06:59:31,953 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
> Proccess agent startup answer, agent id = 0
> 2013-09-17 06:59:31,953 INFO [cloud.agent.Agent] (Agent-Handler-2:null) Set
> agent id 0
> 2013-09-17 06:59:31,960 INFO [cloud.agent.Agent] (Agent-Handler-2:null)
> Startup Response Received: agent id = 0
> 2013-09-17 07:01:06,324 INFO [cloud.agent.Agent]
> (agentRequest-Handler-1:null) Proccess agent ready command, agent id = 17
> 2013-09-17 07:01:06,325 INFO [cloud.agent.Agent]
> (agentRequest-Handler-1:null) Set agent id 17
> 2013-09-17 07:01:06,326 INFO [cloud.agent.Agent]
> (agentRequest-Handler-1:null) Ready command is processed: agent id = 17
> 2013-09-17 07:01:06,576 INFO [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Proccess agent ready command, agent id = 17
> 2013-09-17 07:01:06,577 INFO [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Set agent id 17
> 2013-09-17 07:01:06,585 INFO [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Ready command is processed: agent id = 17
> 2013-09-17 07:39:57,937 WARN [kvm.resource.KVMHAMonitor] (Script-7:null)
> Interrupting script.
> 2013-09-17 07:39:57,940 WARN [kvm.resource.KVMHAMonitor] (Thread-42:null)
> Timed out:
> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> 172.16.16.16 -p /cassandra2 -m /mnt/18c08b3e-251e-325e-b910-3063fa00756f -h
> 172.16.16.14 . Output is:
> 2013-09-17 07:39:57,940 WARN [kvm.resource.KVMHAMonitor] (Thread-42:null)
> write heartbeat failed: timeout, retry: 0
> 2013-09-17 07:40:57,949 WARN [kvm.resource.KVMHAMonitor] (Script-6:null)
> Interrupting script.
> 2013-09-17 07:40:57,950 WARN [kvm.resource.KVMHAMonitor] (Thread-42:null)
> Timed out:
> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> 172.16.16.16 -p /cassandra2 -m /mnt/18c08b3e-251e-325e-b910-3063fa00756f -h
> 172.16.16.14 . Output is:
> 2013-09-17 07:40:57,950 WARN [kvm.resource.KVMHAMonitor] (Thread-42:null)
> write heartbeat failed: timeout, retry: 1
> 2013-09-17 07:41:57,958 WARN [kvm.resource.KVMHAMonitor] (Script-10:null)
> Interrupting script.
> 2013-09-17 07:41:57,958 WARN [kvm.resource.KVMHAMonitor] (Thread-42:null)
> Timed out:
> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> 172.16.16.16 -p /cassandra2 -m /mnt/18c08b3e-251e-325e-b910-3063fa00756f -h
> 172.16.16.14 . Output is:
> 2013-09-17 07:41:57,959 WARN [kvm.resource.KVMHAMonitor] (Thread-42:null)
> write heartbeat failed: timeout, retry: 2
>
>
>> KVM Router - to many ethernet devices created
>> ---------------------------------------------
>>
>> Key: CLOUDSTACK-4690
>> URL:
>> https://issues.apache.org/jira/browse/CLOUDSTACK-4690
>> Project: CloudStack
>> Issue Type: Bug
>> Security Level: Public(Anyone can view this level - this is the
>> default.)
>> Components: KVM
>> Affects Versions: 4.1.1
>> Environment: Centos 6.4
>> Reporter: Paul Edwards
>> Priority: Critical
>> Attachments: management-server.log, router.log
>>
>>
>> We have setup cloudstack with advanced networking. We have a network
>> created 172.16.24.0/23, called news preprod. This has a number of vms
>> created under it. When we initially set this up, we noticed that the
>> router we created with 4 ethernet interfaces. This was unexpected, but
>> didn't seem to be effecting the running of the router, so we didn't worry
>> about it. The interfaces were: eth0 172.16.24.14, eth1 169.254.1.91, and
>> both eth2 and eth2 were given the external ip (xxx.xxx.245.59). We
>> investigated why we were getting 2 externals, but couldn't figure it out.
>> We also configured the router to have a redundant, with keepalived. They
>> were working fine.
>> Now I needed to add another external ip to the network. Did that, and
>> restarted the network. I now have 8 (eight) eth interfaces. The new
>> external ip is NOT one of them. Heres the ifconfig output:
>> root@r-235-VM:~# ifconfig
>> eth0 Link encap:Ethernet HWaddr 02:00:68:e1:00:1a
>> inet addr:172.16.24.14 Bcast:172.16.25.255 Mask:255.255.254.0
>> inet6 addr: fe80::68ff:fee1:1a/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:5485 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:21101 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:325618 (317.9 KiB) TX bytes:1581746 (1.5 MiB)
>> eth1 Link encap:Ethernet HWaddr 0e:00:a9:fe:01:5b
>> inet addr:169.254.1.91 Bcast:169.254.255.255 Mask:255.255.0.0
>> inet6 addr: fe80::c00:a9ff:fefe:15b/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:22118 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:21148 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:3942785 (3.7 MiB) TX bytes:4287970 (4.0 MiB)
>> eth2 Link encap:Ethernet HWaddr 06:16:06:00:01:03
>> inet addr:xxx.xxx.245.59 Bcast:xxx.xxx.245.255
>> Mask:255.255.255.0
>> inet6 addr: fe80::416:6ff:fe00:103/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:17331 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:3930 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:1063053 (1.0 MiB) TX bytes:234546 (229.0 KiB)
>> eth3 Link encap:Ethernet HWaddr 06:d7:ec:00:01:03
>> inet addr:xxx.xxx.245.59 Bcast:xxx.xxx.245.255
>> Mask:255.255.255.0
>> inet6 addr: fe80::4d7:ecff:fe00:103/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:13603 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:792907 (774.3 KiB) TX bytes:704 (704.0 B)
>> eth4 Link encap:Ethernet HWaddr 06:8b:5a:00:01:03
>> inet addr:xxx.xxx.245.59 Bcast:xxx.xxx.245.255
>> Mask:255.255.255.0
>> inet6 addr: fe80::48b:5aff:fe00:103/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:13319 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:779753 (761.4 KiB) TX bytes:704 (704.0 B)
>> eth5 Link encap:Ethernet HWaddr 06:7e:68:00:01:03
>> inet addr:xxx.xxx.245.59 Bcast:xxx.xxx.245.255
>> Mask:255.255.255.0
>> inet6 addr: fe80::47e:68ff:fe00:103/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:13080 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:762111 (744.2 KiB) TX bytes:704 (704.0 B)
>> eth6 Link encap:Ethernet HWaddr 06:7e:68:00:01:03
>> inet addr:xxx.xxx.245.59 Bcast:xxx.xxx.245.255
>> Mask:255.255.255.0
>> inet6 addr: fe80::47e:68ff:fe00:103/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:13086 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:762655 (744.7 KiB) TX bytes:774 (774.0 B)
>> eth7 Link encap:Ethernet HWaddr 06:27:a8:00:01:03
>> inet addr:xxx.xxx.245.59 Bcast:xxx.xxx.245.255
>> Mask:255.255.255.0
>> inet6 addr: fe80::427:a8ff:fe00:103/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:13064 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:761153 (743.3 KiB) TX bytes:704 (704.0 B)
>> eth8 Link encap:Ethernet HWaddr 06:27:a8:00:01:03
>> inet addr:xxx.xxx.245.59 Bcast:xxx.xxx.245.255
>> Mask:255.255.255.0
>> inet6 addr: fe80::427:a8ff:fe00:103/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:13043 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:759287 (741.4 KiB) TX bytes:704 (704.0 B)
>> lo Link encap:Local Loopback
>> inet addr:127.0.0.1 Mask:255.0.0.0
>> inet6 addr: ::1/128 Scope:Host
>> UP LOOPBACK RUNNING MTU:16436 Metric:1
>> RX packets:6 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:0
>> RX bytes:414 (414.0 B) TX bytes:414 (414.0 B)
>> The cloudstack management browser still shows the router as only supposed
>> to have 3 nics.
>> The agent log on the physical host has nothing in it (I was tailing that
>> log when I did this, as I would have thought that the agent has to create
>> the devices for the vm to get them).
>> I have attached the management log from just before to just after. (Time
>> of change was 9:39am). I do not know why the router timestamp is not the
>> same as the host.
>> I have also attached the log of the router restart.
>> The physical server is Centos 6.4, and all the vms that we have created
>> are Centos 6.4. KVM is the hypervisor.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>