[jira] [Created] (CLOUDSTACK-2115) [BasicZone-XenServer] Unable to add host to basic zone that is configured with bridge

["venkata swamybabu budumuru (JIRA)" <ji...@apache.org>]

venkata swamybabu budumuru created CLOUDSTACK-2115:
------------------------------------------------------

             Summary: [BasicZone-XenServer] Unable to add host to basic zone that is configured with bridge
                 Key: CLOUDSTACK-2115
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2115
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: XenServer
    Affects Versions: 4.2.0
         Environment: commit 81e1ba3bb406f9546f0d46ccbfa3fbbb35b762e2
            Reporter: venkata swamybabu budumuru
            Priority: Critical
             Fix For: 4.2.0


Steps to reproduce:

1. Create a basic zone
2. Add a XenServer 6.1 host to CloudStack

Note : before adding, have changed the following

- xe-switch-network-backend bridge
- update sysctl.conf with the following
# Disable *tables rules for bridge traffic to increase performance
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-arptables = 1

- sysctl -p /etc/sysctl.conf


Observations:

(i) It failed to add the host with the following error

2013-04-19 07:24:25,283 DEBUG [agent.transport.Request] (DirectAgent-80:null) Seq 1-1241186308: Processing:  { Ans: , MgmtId: 7280707764394, via: 1, Ver: v1, Flags: 10, [{"ClusterSyncAnswer":{"_clusterId":1,"_newStates":{},"_isExecuted":false,"result":true,"wait":0}}] }
2013-04-19 07:24:25,424 DEBUG [xen.resource.XenServerConnectionPool] (DirectAgent-5:null) XmlRpcException for method: host.call_plugin due to Failed to create input stream: Read timed out.  Reconnecting...retry=1
2013-04-19 07:24:25,424 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-5:null) callHostPlugin failed for cmd: setLinkLocalIP with args brName: xapi1,  due to Failed to create input stream: Read timed out
2013-04-19 07:24:25,424 WARN  [agent.manager.DirectAgentAttache] (DirectAgent-5:null) Seq 1-1241186311: Exception Caught while executing command
com.cloud.utils.exception.CloudRuntimeException: callHostPlugin failed for cmd: setLinkLocalIP with args brName: xapi1,  due to Failed to create input stream: Read timed out
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.callHostPlugin(CitrixResourceBase.java:3971)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.setupLinkLocalNetwork(CitrixResourceBase.java:4651)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3159)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:466)
        at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
        at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:679)

(ii) [root@Rack1Pod1Host11 log]# !cat
cat /etc/xensource/network.conf 
bridge

(iii) Found the following in the SMlog

[17790] 2013-04-19 05:58:59.443668      ['cat', '/etc/xensource/network.conf']
[17790] 2013-04-19 05:58:59.451187        pread SUCCESS
[17790] 2013-04-19 05:58:59.451275      ['ovs-vsctl', 'add-br', 'xapi1']

(iv) the above output proves that though the network.conf has value "bridge", it is trying to fire OVS commands 

(v) After further debugging the /etc/xapi.d/plugins/vmops, found that while reading the network.conf, it ended up reading with a "\n" character which is making it picking the ovs commands.

Here is the snippet of code :

        try:
            cmd = ['cat', '/etc/xensource/network.conf']
            result = util.pread2(cmd)
        except:
            return 'can not cat network.conf'

        if result.lower() == "bridge":   ===> result.lower() is returning "bridge\n" hence this condition is failing, changing this to result.lower().strip() resolved the issue for me.
            try:
                cmd = ["brctl", "addbr", brName]
                txt = util.pread2(cmd)
            except:
                pass

        else:
            try:
                cmd = ["ovs-vsctl", "add-br", brName]
                txt = util.pread2(cmd)
            except:
                pass

Attaching the management-server log, api.log, vmops script, SMlog to the bug.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira