You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by wilderrodrigues <gi...@git.apache.org> on 2015/09/03 10:33:57 UTC

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

GitHub user wilderrodrigues opened a pull request:

    https://github.com/apache/cloudstack/pull/773

    CLOUDSTACK-8759 - Destroying VPC router results in a new unusable VPC router

    Steps we reproduced to find the problem:
    
    1. Stop/Destroy the VPC router
    2. Add a virtual machine to one of the VPC tier - it will trigger a VPC router creation
    3. Router is created, but the NICs are not configured
    
    How to recover without this fix:
    1. Stop/destroy the VPC router and restart the VPC
    
    Side effects: private gateways could be misconfigured.
    
    Root cause:
    
    In the VpcNetworkHelperImpl.configureDefaultNics() method, the guest network nic was added in the map prior to the control and public NICs. The order in the map should not matter, however in the LibvirtComputingResource.createVifs() method, there is a logic that relies on the device index - the array index - in order to create the  control nic. I advise a refactor on the data model in order to be able to identify the NIC type instead of relying in the array index.
    
    An integration test was added to cover the fix:
    * test_vpc_router_nics.py
    
    Environment:
    Management Server running on CentOS 7.1
    KVM host running on CentOS 7.1
    CloudStack Agent/Common 4.6.0-SNAPSHOT
    
    Executing the test:
    
    ```
    nosetests --with-marvin --marvin-config=/data/shared/marvin/mct-zone2-kvm2-ISOLATED.cfg -s -a tags=advanced,required_hardware=true component/test_vpc_router_nics.py
    ```
    
    Remark: during the SSH there might be stack traces on the console due to the connection retry. It takes some time to get the PF rules in place and reach the VMs. So, just let the test run until the end.
    
    ```
    Test results:
    
    Create a vpc with two networks with two vms in each network ... === TestName: test_01_VPC_nics_after_destroy | Status : SUCCESS ===
    ok
    
    ----------------------------------------------------------------------
    Ran 1 test in 774.020s
    
    OK
    /tmp//MarvinLogs/test_vpc_router_nics_VH6E9S/results.txt (END)
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ekholabs/cloudstack fix/vpc_nic-CLOUDSTACK-8759

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/cloudstack/pull/773.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #773
    
----
commit 21df0ffe898530e69db48b2c16f34b3bb0a17c14
Author: wilderrodrigues <wr...@schubergphilis.com>
Date:   2015-09-01T12:31:49Z

    CLOUDSTACK-8759 - The guest nic has to be added after the control nic
    
       - The was an issue introduced on 4.5 with the VPC refactor.
       - Adding a VM to a VPC with no router would create a broken VPC router, with no nics

commit 23d0f3fedaa136a0cd562e9617355ab33e2f1ea0
Author: wilderrodrigues <wr...@schubergphilis.com>
Date:   2015-09-02T07:27:39Z

    CLOUDSTACK-8759 - Adding a marvin test in order to cover the fix
    
       - The test will create a VPC, add 2 tier, 2 VMs, ACL, PF and SSH into the VM
       - Then it will stop the router, destroy the router, add another VM to 1 tier and check that we can reach all the VMs

commit 53c9bb4d387ddc298744879526a40d1d2ca4e394
Author: wilderrodrigues <wr...@schubergphilis.com>
Date:   2015-09-03T07:38:56Z

    CLOUDSTACK-8759 - Fix guets nic allocation
    
       - When stopping/detroying a VPC router and creating a new one via the addition os a VM, the networks are no plugged correctly
       - The ETH1 ends up with the IP of the guest nic, which remains down. This causes routes problems and the VMs are not reachable via the pub IPs

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by koushik-das <gi...@git.apache.org>.
Github user koushik-das commented on the pull request:

    https://github.com/apache/cloudstack/pull/773#issuecomment-138493272
  
    Looks like this has broken regular isolated network. VR is not going to 'up' state as SSH is failing on the link local ip. The order of the interfaces on VR has changed and as a result the public IP is getting configured for ssh as below.
    
    tcp        0      0 10.147.30.243:3922      0.0.0.0:*               LISTEN 



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by remibergsma <gi...@git.apache.org>.
Github user remibergsma commented on the pull request:

    https://github.com/apache/cloudstack/pull/773#issuecomment-137937640
  
    All tests green, merging.
    
    @wilderrodrigues please set CLOUDSTACK-8759 resolved. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by remibergsma <gi...@git.apache.org>.
Github user remibergsma commented on the pull request:

    https://github.com/apache/cloudstack/pull/773#issuecomment-137937224
  
    LGTM Tested with two VPCs, each with a VM and created VPN between them. Then destroyed one router, and stop/started one VM to trigger the recreation of the VPC. That worked, and it came up with the correct nics:
    
    ```
    root@r-7-VM:~# ip a
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
        link/ether 0e:00:a9:fe:00:05 brd ff:ff:ff:ff:ff:ff
        inet 169.254.0.5/16 brd 169.254.255.255 scope global eth0
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
        link/ether 06:7c:92:00:00:17 brd ff:ff:ff:ff:ff:ff
        inet 192.168.23.23/24 brd 192.168.23.255 scope global eth1
    4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
        link/ether 02:00:22:cf:00:03 brd ff:ff:ff:ff:ff:ff
        inet 10.0.1.1/24 brd 10.0.1.255 scope global eth2
    ```
    
    As a double check had a look at the VPN:
    
    ```
    000 #2: "vpn-192.168.23.24/0x1":500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 2485s; newest IPSEC; eroute owner; isakmp#1; idle; import:admin initiate
    000 #2: "vpn-192.168.23.24/0x1" esp.c00d412a@192.168.23.24 esp.79aa41ed@192.168.23.23 tun.0@192.168.23.24 tun.0@192.168.23.23 ref=0 refhim=4294901761
    000 #1: "vpn-192.168.23.24/0x1":500 STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 85228s; newest ISAKMP; lastdpd=-1s(seq in:0 out:0); idle; import:admin initiate
    ```
    
    Pings on their internal IPs from one VM to another across the two VPCs also worked. The service was properly restored after a destroy and that was the main goal. Well done @wilderrodrigues ! Also love the tests you wrote.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by wilderrodrigues <gi...@git.apache.org>.
Github user wilderrodrigues commented on a diff in the pull request:

    https://github.com/apache/cloudstack/pull/773#discussion_r38636486
  
    --- Diff: test/integration/component/test_vpc_router_nics.py ---
    @@ -0,0 +1,449 @@
    +# Licensed to the Apache Software Foundation (ASF) under one
    +# or more contributor license agreements.  See the NOTICE file
    +# distributed with this work for additional information
    +# regarding copyright ownership.  The ASF licenses this file
    +# to you under the Apache License, Version 2.0 (the
    +# "License"); you may not use this file except in compliance
    +# with the License.  You may obtain a copy of the License at
    +#
    +#   http://www.apache.org/licenses/LICENSE-2.0
    +#
    +# Unless required by applicable law or agreed to in writing,
    +# software distributed under the License is distributed on an
    +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    +# KIND, either express or implied.  See the License for the
    +# specific language governing permissions and limitations
    +# under the License.
    +
    +""" Test VPC nics after router is destroyed """
    +
    +from nose.plugins.attrib import attr
    +from marvin.cloudstackTestCase import cloudstackTestCase
    +from marvin.lib.base import (stopRouter,
    +                             startRouter,
    +                             destroyRouter,
    +                             Account,
    +                             VpcOffering,
    +                             VPC,
    +                             ServiceOffering,
    +                             NATRule,
    +                             NetworkACL,
    +                             PublicIPAddress,
    +                             NetworkOffering,
    +                             Network,
    +                             VirtualMachine,
    +                             LoadBalancerRule)
    +from marvin.lib.common import (get_domain,
    +                               get_zone,
    +                               get_template,
    +                               list_routers)
    +from marvin.lib.utils import cleanup_resources
    +import socket
    +import time
    +import inspect
    +
    +
    +class Services:
    +    """Test VPC network services - Port Forwarding Rules Test Data Class.
    +    """
    +
    +    def __init__(self):
    +        self.services = {
    +            "account": {
    +                "email": "test@test.com",
    +                "firstname": "Test",
    +                "lastname": "User",
    +                "username": "test",
    +                # Random characters are appended for unique
    +                # username
    +                "password": "password",
    +            },
    +            "service_offering": {
    +                "name": "Tiny Instance",
    +                "displaytext": "Tiny Instance",
    +                "cpunumber": 1,
    +                "cpuspeed": 100,
    +                "memory": 128,
    +            },
    +            "network_offering": {
    +                "name": 'VPC Network offering',
    +                "displaytext": 'VPC Network off',
    +                "guestiptype": 'Isolated',
    +                "supportedservices": 'Vpn,Dhcp,Dns,SourceNat,PortForwarding,Lb,UserData,StaticNat,NetworkACL',
    +                "traffictype": 'GUEST',
    +                "availability": 'Optional',
    +                "useVpc": 'on',
    +                "serviceProviderList": {
    +                    "Vpn": 'VpcVirtualRouter',
    +                    "Dhcp": 'VpcVirtualRouter',
    +                    "Dns": 'VpcVirtualRouter',
    +                    "SourceNat": 'VpcVirtualRouter',
    +                    "PortForwarding": 'VpcVirtualRouter',
    +                    "Lb": 'VpcVirtualRouter',
    +                    "UserData": 'VpcVirtualRouter',
    +                    "StaticNat": 'VpcVirtualRouter',
    +                    "NetworkACL": 'VpcVirtualRouter'
    +                },
    +            },
    +            "network_offering_no_lb": {
    +                "name": 'VPC Network offering',
    +                "displaytext": 'VPC Network off',
    +                "guestiptype": 'Isolated',
    +                "supportedservices": 'Dhcp,Dns,SourceNat,PortForwarding,UserData,StaticNat,NetworkACL',
    +                "traffictype": 'GUEST',
    +                "availability": 'Optional',
    +                "useVpc": 'on',
    +                "serviceProviderList": {
    +                    "Dhcp": 'VpcVirtualRouter',
    +                    "Dns": 'VpcVirtualRouter',
    +                    "SourceNat": 'VpcVirtualRouter',
    +                    "PortForwarding": 'VpcVirtualRouter',
    +                    "UserData": 'VpcVirtualRouter',
    +                    "StaticNat": 'VpcVirtualRouter',
    +                    "NetworkACL": 'VpcVirtualRouter'
    +                },
    +            },
    +            "vpc_offering": {
    +                "name": 'VPC off',
    +                "displaytext": 'VPC off',
    +                "supportedservices": 'Dhcp,Dns,SourceNat,PortForwarding,Vpn,Lb,UserData,StaticNat',
    +            },
    +            "vpc": {
    +                "name": "TestVPC",
    +                "displaytext": "TestVPC",
    +                "cidr": '10.0.0.1/24'
    +            },
    +            "network": {
    +                "name": "Test Network",
    +                "displaytext": "Test Network",
    +                "netmask": '255.255.255.0'
    +            },
    +            "lbrule": {
    +                "name": "SSH",
    +                "alg": "leastconn",
    +                # Algorithm used for load balancing
    +                "privateport": 22,
    +                "publicport": 2222,
    +                "openfirewall": False,
    +                "startport": 22,
    +                "endport": 2222,
    +                "protocol": "TCP",
    +                "cidrlist": '0.0.0.0/0',
    +            },
    +            "lbrule_http": {
    +                "name": "HTTP",
    +                "alg": "leastconn",
    +                # Algorithm used for load balancing
    +                "privateport": 80,
    +                "publicport": 8888,
    +                "openfirewall": False,
    +                "startport": 80,
    +                "endport": 8888,
    +                "protocol": "TCP",
    +                "cidrlist": '0.0.0.0/0',
    +            },
    +            "natrule": {
    +                "privateport": 22,
    +                "publicport": 22,
    +                "startport": 22,
    +                "endport": 22,
    +                "protocol": "TCP",
    +                "cidrlist": '0.0.0.0/0',
    +            },
    +            "http_rule": {
    +                "privateport": 80,
    +                "publicport": 80,
    +                "startport": 80,
    +                "endport": 80,
    +                "cidrlist": '0.0.0.0/0',
    +                "protocol": "TCP"
    +            },
    +            "virtual_machine": {
    +                "displayname": "Test VM",
    +                "username": "root",
    +                "password": "password",
    +                "ssh_port": 22,
    +                "privateport": 22,
    +                "publicport": 22,
    +                "protocol": 'TCP',
    +            },
    +            "ostype": 'CentOS 5.3 (64-bit)',
    +            "timeout": 10,
    +        }
    +
    +
    +class TestVPCNics(cloudstackTestCase):
    +
    +    @classmethod
    +    def setUpClass(cls):
    +        # We want to fail quicker if it's failure
    +        socket.setdefaulttimeout(60)
    +
    +        cls.testClient = super(TestVPCNics, cls).getClsTestClient()
    +        cls.api_client = cls.testClient.getApiClient()
    +
    +        cls.services = Services().services
    +        # Get Zone, Domain and templates
    +        cls.domain = get_domain(cls.api_client)
    +        cls.zone = get_zone(cls.api_client, cls.testClient.getZoneForTests())
    +        cls.template = get_template(
    +            cls.api_client,
    +            cls.zone.id,
    +            cls.services["ostype"])
    +        cls.services["virtual_machine"]["zoneid"] = cls.zone.id
    +        cls.services["virtual_machine"]["template"] = cls.template.id
    +
    +        cls.service_offering = ServiceOffering.create(
    +            cls.api_client,
    +            cls.services["service_offering"])
    +        cls._cleanup = [cls.service_offering]
    +        return
    +
    +    @classmethod
    +    def tearDownClass(cls):
    +        try:
    +            cleanup_resources(cls.api_client, cls._cleanup)
    +        except Exception as e:
    +            raise Exception("Warning: Exception during cleanup : %s" % e)
    +        return
    +
    +    def setUp(self):
    +        self.routers = []
    +        self.networks = []
    +        self.ips = []
    +        self.apiclient = self.testClient.getApiClient()
    +        self.account = Account.create(
    +            self.apiclient,
    +            self.services["account"],
    +            admin=True,
    +            domainid=self.domain.id)
    +
    +        self.cleanup = [self.account]
    +        self.debug("Creating a VPC offering..")
    +        self.vpc_off = VpcOffering.create(
    +            self.apiclient,
    +            self.services["vpc_offering"])
    +
    +        self.debug("Enabling the VPC offering created")
    +        self.vpc_off.update(self.apiclient, state='Enabled')
    +
    +        self.debug("Creating a VPC network in the account: %s" % self.account.name)
    +        self.services["vpc"]["cidr"] = '10.1.1.1/16'
    +        self.vpc = VPC.create(
    +            self.apiclient,
    +            self.services["vpc"],
    +            vpcofferingid=self.vpc_off.id,
    +            zoneid=self.zone.id,
    +            account=self.account.name,
    +            domainid=self.account.domainid)
    +        return
    +
    +    def tearDown(self):
    +        try:
    +            cleanup_resources(self.apiclient, self.cleanup)
    +        except Exception as e:
    +            self.debug("Warning: Exception during cleanup : %s" % e)
    +        return
    +
    +    def query_routers(self):
    +        self.routers = list_routers(self.apiclient,
    +                                    account=self.account.name,
    +                                    domainid=self.account.domainid,
    +                                    )
    --- End diff --
    
    The test was partially copied form another test, which was copied from test_routers.py. The extra ',' is not needed, indeed. Although it causes no harm. Concerning the parentheses, brackets and curly-brackets alone in the line, that's the style the Marvin tests were written with. For example:
    
    ```
            cls._cleanup = [
                cls.account,
                cls.service_offering
            ]
    ```
    The snippet above can be found at test_routers.py.
    
    I could write Python in a different way, but I don't really mind adapting to their style since all the other tests are written in the same way.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/cloudstack/pull/773


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by karuturi <gi...@git.apache.org>.
Github user karuturi commented on the pull request:

    https://github.com/apache/cloudstack/pull/773#issuecomment-137682596
  
    code :+1: (Busy with something else and cannot test right now)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by wilderrodrigues <gi...@git.apache.org>.
Github user wilderrodrigues commented on the pull request:

    https://github.com/apache/cloudstack/pull/773#issuecomment-137664614
  
    Guys, any other LGTM for this PR?
    
    @remibergsma @karuturi @bhaisaab @DaanHoogland 
    
    Cheers,
    Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by miguelaferreira <gi...@git.apache.org>.
Github user miguelaferreira commented on the pull request:

    https://github.com/apache/cloudstack/pull/773#issuecomment-137412009
  
    There is a typo in message of commit 53c9bb4d387ddc298744879526a40d1d2ca4e394 (guets)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by miguelaferreira <gi...@git.apache.org>.
Github user miguelaferreira commented on a diff in the pull request:

    https://github.com/apache/cloudstack/pull/773#discussion_r38642313
  
    --- Diff: test/integration/component/test_vpc_router_nics.py ---
    @@ -0,0 +1,449 @@
    +# Licensed to the Apache Software Foundation (ASF) under one
    +# or more contributor license agreements.  See the NOTICE file
    +# distributed with this work for additional information
    +# regarding copyright ownership.  The ASF licenses this file
    +# to you under the Apache License, Version 2.0 (the
    +# "License"); you may not use this file except in compliance
    +# with the License.  You may obtain a copy of the License at
    +#
    +#   http://www.apache.org/licenses/LICENSE-2.0
    +#
    +# Unless required by applicable law or agreed to in writing,
    +# software distributed under the License is distributed on an
    +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    +# KIND, either express or implied.  See the License for the
    +# specific language governing permissions and limitations
    +# under the License.
    +
    +""" Test VPC nics after router is destroyed """
    +
    +from nose.plugins.attrib import attr
    +from marvin.cloudstackTestCase import cloudstackTestCase
    +from marvin.lib.base import (stopRouter,
    +                             startRouter,
    +                             destroyRouter,
    +                             Account,
    +                             VpcOffering,
    +                             VPC,
    +                             ServiceOffering,
    +                             NATRule,
    +                             NetworkACL,
    +                             PublicIPAddress,
    +                             NetworkOffering,
    +                             Network,
    +                             VirtualMachine,
    +                             LoadBalancerRule)
    +from marvin.lib.common import (get_domain,
    +                               get_zone,
    +                               get_template,
    +                               list_routers)
    +from marvin.lib.utils import cleanup_resources
    +import socket
    +import time
    +import inspect
    +
    +
    +class Services:
    +    """Test VPC network services - Port Forwarding Rules Test Data Class.
    +    """
    +
    +    def __init__(self):
    +        self.services = {
    +            "account": {
    +                "email": "test@test.com",
    +                "firstname": "Test",
    +                "lastname": "User",
    +                "username": "test",
    +                # Random characters are appended for unique
    +                # username
    +                "password": "password",
    +            },
    +            "service_offering": {
    +                "name": "Tiny Instance",
    +                "displaytext": "Tiny Instance",
    +                "cpunumber": 1,
    +                "cpuspeed": 100,
    +                "memory": 128,
    +            },
    +            "network_offering": {
    +                "name": 'VPC Network offering',
    +                "displaytext": 'VPC Network off',
    +                "guestiptype": 'Isolated',
    +                "supportedservices": 'Vpn,Dhcp,Dns,SourceNat,PortForwarding,Lb,UserData,StaticNat,NetworkACL',
    +                "traffictype": 'GUEST',
    +                "availability": 'Optional',
    +                "useVpc": 'on',
    +                "serviceProviderList": {
    +                    "Vpn": 'VpcVirtualRouter',
    +                    "Dhcp": 'VpcVirtualRouter',
    +                    "Dns": 'VpcVirtualRouter',
    +                    "SourceNat": 'VpcVirtualRouter',
    +                    "PortForwarding": 'VpcVirtualRouter',
    +                    "Lb": 'VpcVirtualRouter',
    +                    "UserData": 'VpcVirtualRouter',
    +                    "StaticNat": 'VpcVirtualRouter',
    +                    "NetworkACL": 'VpcVirtualRouter'
    +                },
    +            },
    +            "network_offering_no_lb": {
    +                "name": 'VPC Network offering',
    +                "displaytext": 'VPC Network off',
    +                "guestiptype": 'Isolated',
    +                "supportedservices": 'Dhcp,Dns,SourceNat,PortForwarding,UserData,StaticNat,NetworkACL',
    +                "traffictype": 'GUEST',
    +                "availability": 'Optional',
    +                "useVpc": 'on',
    +                "serviceProviderList": {
    +                    "Dhcp": 'VpcVirtualRouter',
    +                    "Dns": 'VpcVirtualRouter',
    +                    "SourceNat": 'VpcVirtualRouter',
    +                    "PortForwarding": 'VpcVirtualRouter',
    +                    "UserData": 'VpcVirtualRouter',
    +                    "StaticNat": 'VpcVirtualRouter',
    +                    "NetworkACL": 'VpcVirtualRouter'
    +                },
    +            },
    +            "vpc_offering": {
    +                "name": 'VPC off',
    +                "displaytext": 'VPC off',
    +                "supportedservices": 'Dhcp,Dns,SourceNat,PortForwarding,Vpn,Lb,UserData,StaticNat',
    +            },
    +            "vpc": {
    +                "name": "TestVPC",
    +                "displaytext": "TestVPC",
    +                "cidr": '10.0.0.1/24'
    +            },
    +            "network": {
    +                "name": "Test Network",
    +                "displaytext": "Test Network",
    +                "netmask": '255.255.255.0'
    +            },
    +            "lbrule": {
    +                "name": "SSH",
    +                "alg": "leastconn",
    +                # Algorithm used for load balancing
    +                "privateport": 22,
    +                "publicport": 2222,
    +                "openfirewall": False,
    +                "startport": 22,
    +                "endport": 2222,
    +                "protocol": "TCP",
    +                "cidrlist": '0.0.0.0/0',
    +            },
    +            "lbrule_http": {
    +                "name": "HTTP",
    +                "alg": "leastconn",
    +                # Algorithm used for load balancing
    +                "privateport": 80,
    +                "publicport": 8888,
    +                "openfirewall": False,
    +                "startport": 80,
    +                "endport": 8888,
    +                "protocol": "TCP",
    +                "cidrlist": '0.0.0.0/0',
    +            },
    +            "natrule": {
    +                "privateport": 22,
    +                "publicport": 22,
    +                "startport": 22,
    +                "endport": 22,
    +                "protocol": "TCP",
    +                "cidrlist": '0.0.0.0/0',
    +            },
    +            "http_rule": {
    +                "privateport": 80,
    +                "publicport": 80,
    +                "startport": 80,
    +                "endport": 80,
    +                "cidrlist": '0.0.0.0/0',
    +                "protocol": "TCP"
    +            },
    +            "virtual_machine": {
    +                "displayname": "Test VM",
    +                "username": "root",
    +                "password": "password",
    +                "ssh_port": 22,
    +                "privateport": 22,
    +                "publicport": 22,
    +                "protocol": 'TCP',
    +            },
    +            "ostype": 'CentOS 5.3 (64-bit)',
    +            "timeout": 10,
    +        }
    +
    +
    +class TestVPCNics(cloudstackTestCase):
    +
    +    @classmethod
    +    def setUpClass(cls):
    +        # We want to fail quicker if it's failure
    +        socket.setdefaulttimeout(60)
    +
    +        cls.testClient = super(TestVPCNics, cls).getClsTestClient()
    +        cls.api_client = cls.testClient.getApiClient()
    +
    +        cls.services = Services().services
    +        # Get Zone, Domain and templates
    +        cls.domain = get_domain(cls.api_client)
    +        cls.zone = get_zone(cls.api_client, cls.testClient.getZoneForTests())
    +        cls.template = get_template(
    +            cls.api_client,
    +            cls.zone.id,
    +            cls.services["ostype"])
    +        cls.services["virtual_machine"]["zoneid"] = cls.zone.id
    +        cls.services["virtual_machine"]["template"] = cls.template.id
    +
    +        cls.service_offering = ServiceOffering.create(
    +            cls.api_client,
    +            cls.services["service_offering"])
    +        cls._cleanup = [cls.service_offering]
    +        return
    +
    +    @classmethod
    +    def tearDownClass(cls):
    +        try:
    +            cleanup_resources(cls.api_client, cls._cleanup)
    +        except Exception as e:
    +            raise Exception("Warning: Exception during cleanup : %s" % e)
    +        return
    +
    +    def setUp(self):
    +        self.routers = []
    +        self.networks = []
    +        self.ips = []
    +        self.apiclient = self.testClient.getApiClient()
    +        self.account = Account.create(
    +            self.apiclient,
    +            self.services["account"],
    +            admin=True,
    +            domainid=self.domain.id)
    +
    +        self.cleanup = [self.account]
    +        self.debug("Creating a VPC offering..")
    +        self.vpc_off = VpcOffering.create(
    +            self.apiclient,
    +            self.services["vpc_offering"])
    +
    +        self.debug("Enabling the VPC offering created")
    +        self.vpc_off.update(self.apiclient, state='Enabled')
    +
    +        self.debug("Creating a VPC network in the account: %s" % self.account.name)
    +        self.services["vpc"]["cidr"] = '10.1.1.1/16'
    +        self.vpc = VPC.create(
    +            self.apiclient,
    +            self.services["vpc"],
    +            vpcofferingid=self.vpc_off.id,
    +            zoneid=self.zone.id,
    +            account=self.account.name,
    +            domainid=self.account.domainid)
    +        return
    +
    +    def tearDown(self):
    +        try:
    +            cleanup_resources(self.apiclient, self.cleanup)
    +        except Exception as e:
    +            self.debug("Warning: Exception during cleanup : %s" % e)
    +        return
    +
    +    def query_routers(self):
    +        self.routers = list_routers(self.apiclient,
    +                                    account=self.account.name,
    +                                    domainid=self.account.domainid,
    +                                    )
    --- End diff --
    
    My point is that in this file you use two different styles.
    
    This one (with the last ')' in a separate line):
    ```
            self.assertEqual(
                isinstance(router_response, list),
                True,
                "Check list response returns a valid list"
            )
    ```
    
    And this one (with the last ')' inline) as well:
    ```
            self.assertEqual(
                router.hostid, host.id, "Migration to host %s failed. The router host is"
                " still %s" %
                (host.id, router.hostid))
    ```
    
    One way or another is fine as long as it is consistent. Don't you agree?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by wilderrodrigues <gi...@git.apache.org>.
Github user wilderrodrigues commented on the pull request:

    https://github.com/apache/cloudstack/pull/773#issuecomment-138523786
  
    Hi @koushik-das ,
    
    I will have a look into this.
    
    Cheers,
    Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by miguelaferreira <gi...@git.apache.org>.
Github user miguelaferreira commented on a diff in the pull request:

    https://github.com/apache/cloudstack/pull/773#discussion_r38634904
  
    --- Diff: test/integration/component/test_vpc_router_nics.py ---
    @@ -0,0 +1,449 @@
    +# Licensed to the Apache Software Foundation (ASF) under one
    +# or more contributor license agreements.  See the NOTICE file
    +# distributed with this work for additional information
    +# regarding copyright ownership.  The ASF licenses this file
    +# to you under the Apache License, Version 2.0 (the
    +# "License"); you may not use this file except in compliance
    +# with the License.  You may obtain a copy of the License at
    +#
    +#   http://www.apache.org/licenses/LICENSE-2.0
    +#
    +# Unless required by applicable law or agreed to in writing,
    +# software distributed under the License is distributed on an
    +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    +# KIND, either express or implied.  See the License for the
    +# specific language governing permissions and limitations
    +# under the License.
    +
    +""" Test VPC nics after router is destroyed """
    +
    +from nose.plugins.attrib import attr
    +from marvin.cloudstackTestCase import cloudstackTestCase
    +from marvin.lib.base import (stopRouter,
    +                             startRouter,
    +                             destroyRouter,
    +                             Account,
    +                             VpcOffering,
    +                             VPC,
    +                             ServiceOffering,
    +                             NATRule,
    +                             NetworkACL,
    +                             PublicIPAddress,
    +                             NetworkOffering,
    +                             Network,
    +                             VirtualMachine,
    +                             LoadBalancerRule)
    +from marvin.lib.common import (get_domain,
    +                               get_zone,
    +                               get_template,
    +                               list_routers)
    +from marvin.lib.utils import cleanup_resources
    +import socket
    +import time
    +import inspect
    +
    +
    +class Services:
    +    """Test VPC network services - Port Forwarding Rules Test Data Class.
    +    """
    +
    +    def __init__(self):
    +        self.services = {
    +            "account": {
    +                "email": "test@test.com",
    +                "firstname": "Test",
    +                "lastname": "User",
    +                "username": "test",
    +                # Random characters are appended for unique
    +                # username
    +                "password": "password",
    +            },
    +            "service_offering": {
    +                "name": "Tiny Instance",
    +                "displaytext": "Tiny Instance",
    +                "cpunumber": 1,
    +                "cpuspeed": 100,
    +                "memory": 128,
    +            },
    +            "network_offering": {
    +                "name": 'VPC Network offering',
    +                "displaytext": 'VPC Network off',
    +                "guestiptype": 'Isolated',
    +                "supportedservices": 'Vpn,Dhcp,Dns,SourceNat,PortForwarding,Lb,UserData,StaticNat,NetworkACL',
    +                "traffictype": 'GUEST',
    +                "availability": 'Optional',
    +                "useVpc": 'on',
    +                "serviceProviderList": {
    +                    "Vpn": 'VpcVirtualRouter',
    +                    "Dhcp": 'VpcVirtualRouter',
    +                    "Dns": 'VpcVirtualRouter',
    +                    "SourceNat": 'VpcVirtualRouter',
    +                    "PortForwarding": 'VpcVirtualRouter',
    +                    "Lb": 'VpcVirtualRouter',
    +                    "UserData": 'VpcVirtualRouter',
    +                    "StaticNat": 'VpcVirtualRouter',
    +                    "NetworkACL": 'VpcVirtualRouter'
    +                },
    +            },
    +            "network_offering_no_lb": {
    +                "name": 'VPC Network offering',
    +                "displaytext": 'VPC Network off',
    +                "guestiptype": 'Isolated',
    +                "supportedservices": 'Dhcp,Dns,SourceNat,PortForwarding,UserData,StaticNat,NetworkACL',
    +                "traffictype": 'GUEST',
    +                "availability": 'Optional',
    +                "useVpc": 'on',
    +                "serviceProviderList": {
    +                    "Dhcp": 'VpcVirtualRouter',
    +                    "Dns": 'VpcVirtualRouter',
    +                    "SourceNat": 'VpcVirtualRouter',
    +                    "PortForwarding": 'VpcVirtualRouter',
    +                    "UserData": 'VpcVirtualRouter',
    +                    "StaticNat": 'VpcVirtualRouter',
    +                    "NetworkACL": 'VpcVirtualRouter'
    +                },
    +            },
    +            "vpc_offering": {
    +                "name": 'VPC off',
    +                "displaytext": 'VPC off',
    +                "supportedservices": 'Dhcp,Dns,SourceNat,PortForwarding,Vpn,Lb,UserData,StaticNat',
    +            },
    +            "vpc": {
    +                "name": "TestVPC",
    +                "displaytext": "TestVPC",
    +                "cidr": '10.0.0.1/24'
    +            },
    +            "network": {
    +                "name": "Test Network",
    +                "displaytext": "Test Network",
    +                "netmask": '255.255.255.0'
    +            },
    +            "lbrule": {
    +                "name": "SSH",
    +                "alg": "leastconn",
    +                # Algorithm used for load balancing
    +                "privateport": 22,
    +                "publicport": 2222,
    +                "openfirewall": False,
    +                "startport": 22,
    +                "endport": 2222,
    +                "protocol": "TCP",
    +                "cidrlist": '0.0.0.0/0',
    +            },
    +            "lbrule_http": {
    +                "name": "HTTP",
    +                "alg": "leastconn",
    +                # Algorithm used for load balancing
    +                "privateport": 80,
    +                "publicport": 8888,
    +                "openfirewall": False,
    +                "startport": 80,
    +                "endport": 8888,
    +                "protocol": "TCP",
    +                "cidrlist": '0.0.0.0/0',
    +            },
    +            "natrule": {
    +                "privateport": 22,
    +                "publicport": 22,
    +                "startport": 22,
    +                "endport": 22,
    +                "protocol": "TCP",
    +                "cidrlist": '0.0.0.0/0',
    +            },
    +            "http_rule": {
    +                "privateport": 80,
    +                "publicport": 80,
    +                "startport": 80,
    +                "endport": 80,
    +                "cidrlist": '0.0.0.0/0',
    +                "protocol": "TCP"
    +            },
    +            "virtual_machine": {
    +                "displayname": "Test VM",
    +                "username": "root",
    +                "password": "password",
    +                "ssh_port": 22,
    +                "privateport": 22,
    +                "publicport": 22,
    +                "protocol": 'TCP',
    +            },
    +            "ostype": 'CentOS 5.3 (64-bit)',
    +            "timeout": 10,
    +        }
    +
    +
    +class TestVPCNics(cloudstackTestCase):
    +
    +    @classmethod
    +    def setUpClass(cls):
    +        # We want to fail quicker if it's failure
    +        socket.setdefaulttimeout(60)
    +
    +        cls.testClient = super(TestVPCNics, cls).getClsTestClient()
    +        cls.api_client = cls.testClient.getApiClient()
    +
    +        cls.services = Services().services
    +        # Get Zone, Domain and templates
    +        cls.domain = get_domain(cls.api_client)
    +        cls.zone = get_zone(cls.api_client, cls.testClient.getZoneForTests())
    +        cls.template = get_template(
    +            cls.api_client,
    +            cls.zone.id,
    +            cls.services["ostype"])
    +        cls.services["virtual_machine"]["zoneid"] = cls.zone.id
    +        cls.services["virtual_machine"]["template"] = cls.template.id
    +
    +        cls.service_offering = ServiceOffering.create(
    +            cls.api_client,
    +            cls.services["service_offering"])
    +        cls._cleanup = [cls.service_offering]
    +        return
    +
    +    @classmethod
    +    def tearDownClass(cls):
    +        try:
    +            cleanup_resources(cls.api_client, cls._cleanup)
    +        except Exception as e:
    +            raise Exception("Warning: Exception during cleanup : %s" % e)
    +        return
    +
    +    def setUp(self):
    +        self.routers = []
    +        self.networks = []
    +        self.ips = []
    +        self.apiclient = self.testClient.getApiClient()
    +        self.account = Account.create(
    +            self.apiclient,
    +            self.services["account"],
    +            admin=True,
    +            domainid=self.domain.id)
    +
    +        self.cleanup = [self.account]
    +        self.debug("Creating a VPC offering..")
    +        self.vpc_off = VpcOffering.create(
    +            self.apiclient,
    +            self.services["vpc_offering"])
    +
    +        self.debug("Enabling the VPC offering created")
    +        self.vpc_off.update(self.apiclient, state='Enabled')
    +
    +        self.debug("Creating a VPC network in the account: %s" % self.account.name)
    +        self.services["vpc"]["cidr"] = '10.1.1.1/16'
    +        self.vpc = VPC.create(
    +            self.apiclient,
    +            self.services["vpc"],
    +            vpcofferingid=self.vpc_off.id,
    +            zoneid=self.zone.id,
    +            account=self.account.name,
    +            domainid=self.account.domainid)
    +        return
    +
    +    def tearDown(self):
    +        try:
    +            cleanup_resources(self.apiclient, self.cleanup)
    +        except Exception as e:
    +            self.debug("Warning: Exception during cleanup : %s" % e)
    +        return
    +
    +    def query_routers(self):
    +        self.routers = list_routers(self.apiclient,
    +                                    account=self.account.name,
    +                                    domainid=self.account.domainid,
    +                                    )
    --- End diff --
    
    I would leave this ')' at the end of the previous line (which by the way has a ',' too many).
    That would be inline with the rest of the code in this method


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by miguelaferreira <gi...@git.apache.org>.
Github user miguelaferreira commented on the pull request:

    https://github.com/apache/cloudstack/pull/773#issuecomment-137437571
  
    My testing:
    ```
    > mvn clean install -P developer,systemvm 
    ...
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 8:39.748s
    [INFO] Finished at: Thu Sep 03 11:44:23 GMT 2015
    [INFO] Final Memory: 94M/412M
    [INFO] ------------------------------------------------------------------------
    
    
    > mvn -P developer -pl developer -Ddeploydb
    ...
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 19.038s
    [INFO] Finished at: Thu Sep 03 11:56:29 GMT 2015
    [INFO] Final Memory: 47M/254M
    [INFO] ------------------------------------------------------------------------
    
    
    > bash -x ./scripts/storage/secondary/cloud-install-sys-tmplt -m /data/storage/secondary/MCCT-SHARED-2/ -u http://jenkins.buildacloud.org/job/build-systemvm64-master/lastSuccessfulBuild/artifact/tools/appliance/dist/systemvm64template-master-4.6.0-kvm.qcow2.bz2 -h kvm -o localhost -r root -e qcow2 -F
    ...
    Successfully installed system VM template  to /data/storage/secondary/MCCT-SHARED-2/template/tmpl/1/3/
    
    
    > mvn -pl :cloud-client-ui jetty:run 
    ...
    Management server node 127.0.0.1 is up
    ...
    
    
    > pip install --upgrade tools/marvin/dist/Marvin-4.6.0-SNAPSHOT.tar.gz
    ...
    Successfully installed Marvin-4.6.0-SNAPSHOT ddt-1.0.0 ecdsa-0.13 netaddr-0.7.17 paramiko-1.15.2 pyvmomi-5.5.0.2014.1.1 requests-2.7.0 six-1.9.0
    
    
    > python tools/marvin/marvin/deployDataCenter.py -i  /data/shared/marvin/mct-zone2-kvm2-ISOLATED.cfg
    ...
    ====Deploy DC Successful=====
    
    
    > nosetests --with-marvin --marvin-config=/data/shared/marvin/mct-zone2-kvm2-ISOLATED.cfg -s -a tags=advanced,required_hardware=true component/test_vpc_router_nics.py
    ...
    === TestName: test_01_VPC_nics_after_destroy | Status : SUCCESS ===
    
    ===final results are now copied to: /tmp//MarvinLogs/test_vpc_router_nics_NAU8EJ===
    ```
    
    :+1: 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8759 - Destroying VPC router r...

Posted by wilderrodrigues <gi...@git.apache.org>.
Github user wilderrodrigues commented on the pull request:

    https://github.com/apache/cloudstack/pull/773#issuecomment-137378923
  
    Ping @remibergsma @miguelaferreira @DaanHoogland @bhaisaab @karuturi 
    
    I would appreciate if you would have some time to test this PR.
    
    Thanks in advance.
    
    Cheers,
    Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---