You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by wilderrodrigues <gi...@git.apache.org> on 2015/10/11 15:02:22 UTC

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

GitHub user wilderrodrigues opened a pull request:

    https://github.com/apache/cloudstack/pull/923

    CLOUDSTACK-8934 - Default routes not configured for rVPC and RVR

    This PR fixes the default routes for redundant VPCs and isolated networks. New tests were introduced in order to make sure that the routers are working properly.
    
    During the tests, I found out that the Firewall Egress was not working properly when creating the network offering with default allow. The bug has been fixed and tests for redundant and non-redundant isolated networks were added.
    
    Test reports will follow in a separate comment.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ekholabs/cloudstack fix/default_routes_CLOUDSTACK-8934

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/cloudstack/pull/923.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #923
    
----
commit 45642b83821ce0ecd6d4cddb76a77a2481e54d9a
Author: Wilder Rodrigues <wr...@schubergphilis.com>
Date:   2015-10-08T12:19:26Z

    CLOUDSTACK-8934 - Add default gateway when the public interface is up again

commit 5d1cdc64409caed239f6814a8f73266c8e5eb6fe
Author: Wilder Rodrigues <wr...@schubergphilis.com>
Date:   2015-10-09T11:31:32Z

    CLOUDSTACK-8934 - Adding tests to cover default routes on IsoNest and RVR nets

commit b4dc392bfdf4fb93e1652203b7d4027651cca5ac
Author: Wilder Rodrigues <wr...@schubergphilis.com>
Date:   2015-10-09T12:32:35Z

    CLOUDSTACK-8934 - Fix default EGRESS rules for isolated networks
    
       - The default is Accept and will be changed based on the configuration of the offering.
    
    CLOUDSTACK-8934 - The default egress is set as Deny in the router.
    
       - We had to change it on the Java side in order to make the apply it once the default is defined as allowed on the net offering

commit cd8c3e97f594639292075d9a704dc7c382a103ca
Author: Wilder Rodrigues <wr...@schubergphilis.com>
Date:   2015-10-10T10:02:47Z

    CLOUDSTACK-8934 - Fix the AclIP class to make it configure the default FW policy

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by wilderrodrigues <gi...@git.apache.org>.
Github user wilderrodrigues commented on the pull request:

    https://github.com/apache/cloudstack/pull/923#issuecomment-147191452
  
    Ping @remibergsma @karuturi @DaanHoogland @miguelaferreira @bvbharat @wido @borisroman 
    
    All, the tests were executed successfully! We still have the problems with cleaning the networks, as you can see below. For that there is an issue which I will work on next (https://issues.apache.org/jira/browse/CLOUDSTACK-8935)
    
    Please, have a look and let me know if it all looks okay.
    
    Cheers,
    Wilder
    
    == Hardware required tests ==
    
    * Management Server + MySQL running on CentOS 7.1
    * One KVM host running on CentOS 7.1
    * ACS Agent and Common RPMs built from source
    
    ````
    Stop existing router, add a PF rule and check we can access the VM ... === TestName: test_isolate_network_FW_PF_default_routes | Status : SUCCESS ===
    ok
    Test redundant router internals ... === TestName: test_RVR_Network_FW_PF_SSH_default_routes | Status : SUCCESS ===
    ok
    Create a redundant VPC with two networks with two VMs in each network ... === TestName: test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | Status : SUCCESS ===
    ok
    Create a redundant VPC with two networks with two VMs in each network and check default routes ... === TestName: test_02_redundant_VPC_default_routes | Status : SUCCESS ===
    ok
    Test iptables default INPUT/FORWARD policy on RouterVM ... === TestName: test_02_routervm_iptables_policies | Status : SUCCESS ===
    ok
    Test iptables default INPUT/FORWARD policies on VPC router ... === TestName: test_01_single_VPC_iptables_policies | Status : SUCCESS ===
    ok
    Create a VPC with two networks with one VM in each network and test nics after destroy ... === TestName: test_01_VPC_nics_after_destroy | Status : SUCCESS ===
    ok
    Create a VPC with two networks with one VM in each network and test default routes ... === TestName: test_02_VPC_default_routes | Status : SUCCESS ===
    ok
    
    ----------------------------------------------------------------------
    Ran 8 tests in 6855.736s
    
    OK
    ````
    
    == No Hardware required tests ==
    
    * Management Server + MySQL running on CentOS 7.1
    * Two KVM hosts running on CentOS 7.1
    * ACS Agent and Common RPMs built from source
    
    ````
    Test advanced zone virtual router ... === TestName: test_advZoneVirtualRouter | Status : SUCCESS ===
    ok
    Test Deploy Virtual Machine ... === TestName: test_deploy_vm | Status : SUCCESS ===
    ok
    Test Multiple Deploy Virtual Machine ... === TestName: test_deploy_vm_multiple | Status : SUCCESS ===
    ok
    Test Stop Virtual Machine ... === TestName: test_01_stop_vm | Status : SUCCESS ===
    ok
    Test Start Virtual Machine ... === TestName: test_02_start_vm | Status : SUCCESS ===
    ok
    Test Reboot Virtual Machine ... === TestName: test_03_reboot_vm | Status : SUCCESS ===
    ok
    Test destroy Virtual Machine ... === TestName: test_06_destroy_vm | Status : SUCCESS ===
    ok
    Test recover Virtual Machine ... === TestName: test_07_restore_vm | Status : SUCCESS ===
    ok
    Test migrate VM ... === TestName: test_08_migrate_vm | Status : SUCCESS ===
    ok
    Test destroy(expunge) Virtual Machine ... === TestName: test_09_expunge_vm | Status : SUCCESS ===
    ok
    Test router internal advanced zone ... === TestName: test_02_router_internal_adv | Status : SUCCESS ===
    ok
    Test restart network ... === TestName: test_03_restart_network_cleanup | Status : SUCCESS ===
    ok
    Test router basic setup ... === TestName: test_05_router_basic | Status : SUCCESS ===
    ok
    Test router advanced setup ... === TestName: test_06_router_advanced | Status : SUCCESS ===
    ok
    Test stop router ... === TestName: test_07_stop_router | Status : SUCCESS ===
    ok
    Test start router ... === TestName: test_08_start_router | Status : SUCCESS ===
    ok
    Test reboot router ... === TestName: test_09_reboot_router | Status : SUCCESS ===
    ok
    Test reset virtual machine on reboot ... === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS ===
    ok
    Test router internal advanced zone ... === TestName: test_02_router_internal_adv | Status : SUCCESS ===
    ok
    Test restart network ... === TestName: test_03_restart_network_cleanup | Status : SUCCESS ===
    ok
    Test router basic setup ... === TestName: test_05_router_basic | Status : SUCCESS ===
    ok
    Test router advanced setup ... === TestName: test_06_router_advanced | Status : SUCCESS ===
    ok
    Test stop router ... === TestName: test_07_stop_router | Status : SUCCESS ===
    ok
    Test start router ... === TestName: test_08_start_router | Status : SUCCESS ===
    ok
    Test reboot router ... === TestName: test_09_reboot_router | Status : SUCCESS ===
    ok
    test_privategw_acl (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_privategw_acl | Status : SUCCESS ===
    ok
    Test VPN in VPC ... === TestName: test_vpc_remote_access_vpn | Status : SUCCESS ===
    ok
    Test VPN in VPC ... === TestName: test_vpc_site2site_vpn | Status : SUCCESS ===
    ok
    Test to create service offering ... === TestName: test_01_create_service_offering | Status : SUCCESS ===
    ok
    Test to update existing service offering ... === TestName: test_02_edit_service_offering | Status : SUCCESS ===
    ok
    Test to delete service offering ... === TestName: test_03_delete_service_offering | Status : SUCCESS ===
    ok
    Test create VPC offering ... === TestName: test_01_create_vpc_offering | Status : SUCCESS ===
    ok
    Test VPC offering without load balancing service ... === TestName: test_03_vpc_off_without_lb | Status : EXCEPTION ===
    ERROR: Test VPC offering without load balancing service
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/data/git/cs2/cloudstack/test/integration/component/test_vpc_offerings.py", line 218, in tearDown
        raise Exception("Warning: Exception during cleanup : %s" % e)
    Exception: Warning: Exception during cleanup : Execute cmd: deletenetworkoffering failed, due to: errorCode: 431, errorText:Can't delete network offering 15 as its used by 1 networks. To make the network offering unavaiable, disable it
    
    
    
    Test VPC offering without static NAT service ... === TestName: test_04_vpc_off_without_static_nat | Status : EXCEPTION ===
    ERROR: Test VPC offering without static NAT service
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/data/git/cs2/cloudstack/test/integration/component/test_vpc_offerings.py", line 218, in tearDown
        raise Exception("Warning: Exception during cleanup : %s" % e)
    Exception: Warning: Exception during cleanup : Execute cmd: deletenetworkoffering failed, due to: errorCode: 431, errorText:Can't delete network offering 16 as its used by 1 networks. To make the network offering unavaiable, disable it
    
    
    
    Test VPC offering without port forwarding service ... === TestName: test_05_vpc_off_without_pf | Status : EXCEPTION ===
    ERROR: Test VPC offering without port forwarding service
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/data/git/cs2/cloudstack/test/integration/component/test_vpc_offerings.py", line 218, in tearDown
        raise Exception("Warning: Exception during cleanup : %s" % e)
    Exception: Warning: Exception during cleanup : Execute cmd: deletenetworkoffering failed, due to: errorCode: 431, errorText:Can't delete network offering 17 as its used by 1 networks. To make the network offering unavaiable, disable it
    
    
    
    Test VPC offering with invalid services ... === TestName: test_06_vpc_off_invalid_services | Status : SUCCESS ===
    ok
    Test update VPC offering ... === TestName: test_07_update_vpc_off | Status : SUCCESS ===
    ok
    Test list VPC offering ... === TestName: test_08_list_vpc_off | Status : SUCCESS ===
    ok
    test_09_create_redundant_vpc_offering (integration.component.test_vpc_offerings.TestVPCOffering) ... === TestName: test_09_create_redundant_vpc_offering | Status : SUCCESS ===
    ok
    Test start/stop of router after addition of one guest network ... === TestName: test_01_start_stop_router_after_addition_of_one_guest_network | Status : SUCCESS ===
    ok
    Test reboot of router after addition of one guest network ... === TestName: test_02_reboot_router_after_addition_of_one_guest_network | Status : SUCCESS ===
    ok
    Test to change service offering of router after addition of one guest network ... === TestName: test_04_chg_srv_off_router_after_addition_of_one_guest_network | Status : SUCCESS ===
    ok
    Test destroy of router after addition of one guest network ... === TestName: test_05_destroy_router_after_addition_of_one_guest_network | Status : SUCCESS ===
    ok
    Test to stop and start router after creation of VPC ... === TestName: test_01_stop_start_router_after_creating_vpc | Status : SUCCESS ===
    ok
    Test to reboot the router after creating a VPC ... === TestName: test_02_reboot_router_after_creating_vpc | Status : SUCCESS ===
    ok
    Tests to change service offering of the Router after ... === TestName: test_04_change_service_offerring_vpc | Status : SUCCESS ===
    ok
    Test to destroy the router after creating a VPC ... === TestName: test_05_destroy_router_after_creating_vpc | Status : SUCCESS ===
    ok
    
    =====================================================================
    
    ----------------------------------------------------------------------
    Ran 47 tests in 8317,912s
    ````


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by wilderrodrigues <gi...@git.apache.org>.
Github user wilderrodrigues commented on a diff in the pull request:

    https://github.com/apache/cloudstack/pull/923#discussion_r41730481
  
    --- Diff: systemvm/patches/debian/config/opt/cloud/bin/configure.py ---
    @@ -123,24 +129,29 @@ def add_rule(self, cidr):
                                         " -p %s " % rule['protocol'] +
                                         " -m %s " % rule['protocol'] +
                                         " --dport %s -j RETURN" % rnge])
    +
    +            logging.debug("Current ACL IP direction is ==> %s", self.direction)
                 if self.direction == 'egress':
    -                self.fw.append(["filter", "", " -A FW_OUTBOUND -j FIREWALL_EGRESS_RULES"])
    +                self.fw.append(["filter", "", " -A FW_OUTBOUND -j FW_EGRESS_RULES"])
    --- End diff --
    
    No, I renamed to make it consistent with the default rules, which can be found in the iptables-router file. That file is installed with cloud early config then afterwards the rules are replaced.
    
    In addition, the code was also refering to FW_OUTBOUND, not FIREWALL_OUTBOUND. The use of FW cs FIREWALL doens't look consistent. Now we have only FW.
    
    I hope it's clear. I am at home today... A bit ill. So typing from my mobile
    
    Cheers,
    Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by DaanHoogland <gi...@git.apache.org>.
Github user DaanHoogland commented on the pull request:

    https://github.com/apache/cloudstack/pull/923#issuecomment-147205805
  
    overall code looks good, not sure if I need to test something special, @wilderrodrigues 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by remibergsma <gi...@git.apache.org>.
Github user remibergsma commented on the pull request:

    https://github.com/apache/cloudstack/pull/923#issuecomment-147342723
  
    @wilderrodrigues I will spend some time on testing this today!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by borisroman <gi...@git.apache.org>.
Github user borisroman commented on the pull request:

    https://github.com/apache/cloudstack/pull/923#issuecomment-147841813
  
    @remibergsma @karuturi @wilderrodrigues @DaanHoogland I tested this PR today. Tests ran fine except for the cleanups (which are known by CLOUDSTACK-8935). I'll spare the space of this comment as all test results already have been posted twice by Wilder and Remi. I also ran the test_loadbalance.py which turned out positive.
    
    Test to create Load balancing rule with source NAT ... === TestName: test_01_create_lb_rule_src_nat | Status : SUCCESS ===
    ok
    Test to create Load balancing rule with non source NAT ... === TestName: test_02_create_lb_rule_non_nat | Status : SUCCESS ===
    ok
    Test for assign & removing load balancing rule ... === TestName: test_assign_and_removal_lb | Status : SUCCESS ===
    ok
    
    ----------------------------------------------------------------------
    Ran 3 tests in 633.239s
    
    OK
    /tmp//MarvinLogs/test_loadbalance_HD34OP/results.txt 
    
    Code (style) changes look good to me.
    
    To conclude: LGTM :+1: 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/cloudstack/pull/923


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by remibergsma <gi...@git.apache.org>.
Github user remibergsma commented on the pull request:

    https://github.com/apache/cloudstack/pull/923#issuecomment-147780057
  
    Let's wait for the test results of @borisroman as he said he was testing as well. 
    @borisroman if you are no longer testing, please let us know.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by DaanHoogland <gi...@git.apache.org>.
Github user DaanHoogland commented on a diff in the pull request:

    https://github.com/apache/cloudstack/pull/923#discussion_r41711315
  
    --- Diff: systemvm/patches/debian/config/opt/cloud/bin/configure.py ---
    @@ -123,24 +129,29 @@ def add_rule(self, cidr):
                                         " -p %s " % rule['protocol'] +
                                         " -m %s " % rule['protocol'] +
                                         " --dport %s -j RETURN" % rnge])
    +
    +            logging.debug("Current ACL IP direction is ==> %s", self.direction)
                 if self.direction == 'egress':
    -                self.fw.append(["filter", "", " -A FW_OUTBOUND -j FIREWALL_EGRESS_RULES"])
    +                self.fw.append(["filter", "", " -A FW_OUTBOUND -j FW_EGRESS_RULES"])
    --- End diff --
    
    is this rename because of an ipfw change?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by borisroman <gi...@git.apache.org>.
Github user borisroman commented on the pull request:

    https://github.com/apache/cloudstack/pull/923#issuecomment-147633618
  
    Testing now, will post results later today!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by karuturi <gi...@git.apache.org>.
Github user karuturi commented on the pull request:

    https://github.com/apache/cloudstack/pull/923#issuecomment-147749011
  
    thanks for running the tests @remibergsma 
    should we merge this PR since we have two reviews from @DaanHoogland and @remibergsma ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: CLOUDSTACK-8934 - Default routes not conf...

Posted by remibergsma <gi...@git.apache.org>.
Github user remibergsma commented on the pull request:

    https://github.com/apache/cloudstack/pull/923#issuecomment-147700296
  
    @wilderrodrigues Thanks, looking good! LGTM
    
    Tested on 2-node KVM cluster on CentOS 7.
    
    I started by running the same tests:
    ```
    nosetests --with-marvin --marvin-config=${marvinCfg} -s -a tags=advanced,required_hardware=true \
    component/test_vpc_redundant.py \
    component/test_routers_iptables_default_policy.py \
    component/test_routers_network_ops.py \
    component/test_vpc_router_nics.py
    ```
    Results:
    ```
    Create a redundant VPC with two networks with two VMs in each network ... === TestName: test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | Status : SUCCESS ===
    ok
    Create a redundant VPC with two networks with two VMs in each network and check default routes ... === TestName: test_02_redundant_VPC_default_routes | Status : SUCCESS ===
    ok
    Stop existing router, add a PF rule and check we can access the VM ... === TestName: test_isolate_network_FW_PF_default_routes | Status : SUCCESS ===
    ok
    Test redundant router internals ... === TestName: test_RVR_Network_FW_PF_SSH_default_routes | Status : SUCCESS ===
    ok
    
    ----------------------------------------------------------------------
    Ran 4 tests in 4592.786s
    
    OK
    ```
    I have seen these tests failing, but trying again and even doing it manually works.
    
    Next test:
    ```
    nosetests --with-marvin --marvin-config=${marvinCfg} -s -a tags=advanced,required_hardware=false \
    smoke/test_routers.py \
    smoke/test_network_acl.py \
    smoke/test_privategw_acl.py \
    smoke/test_reset_vm_on_reboot.py \
    smoke/test_vm_life_cycle.py \
    smoke/test_vpc_vpn.py \
    smoke/test_service_offerings.py \
    component/test_vpc_offerings.py \
    component/test_vpc_routers.py
    ```
    
    Results:
    ```
    Test router internal advanced zone ... === TestName: test_02_router_internal_adv | Status : SUCCESS ===
    ok
    Test restart network ... === TestName: test_03_restart_network_cleanup | Status : SUCCESS ===
    ok
    Test router basic setup ... === TestName: test_05_router_basic | Status : SUCCESS ===
    ok
    Test router advanced setup ... === TestName: test_06_router_advanced | Status : SUCCESS ===
    ok
    Test stop router ... === TestName: test_07_stop_router | Status : SUCCESS ===
    ok
    Test start router ... === TestName: test_08_start_router | Status : SUCCESS ===
    ok
    Test reboot router ... === TestName: test_09_reboot_router | Status : SUCCESS ===
    ok
    test_privategw_acl (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_privategw_acl | Status : SUCCESS ===
    ok
    Test reset virtual machine on reboot ... === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS ===
    ok
    Test advanced zone virtual router ... === TestName: test_advZoneVirtualRouter | Status : SUCCESS ===
    ok
    Test Deploy Virtual Machine ... === TestName: test_deploy_vm | Status : SUCCESS ===
    ok
    Test Multiple Deploy Virtual Machine ... === TestName: test_deploy_vm_multiple | Status : SUCCESS ===
    ok
    Test Stop Virtual Machine ... === TestName: test_01_stop_vm | Status : SUCCESS ===
    ok
    Test Start Virtual Machine ... === TestName: test_02_start_vm | Status : SUCCESS ===
    ok
    Test Reboot Virtual Machine ... === TestName: test_03_reboot_vm | Status : SUCCESS ===
    ok
    Test destroy Virtual Machine ... === TestName: test_06_destroy_vm | Status : SUCCESS ===
    ok
    Test recover Virtual Machine ... === TestName: test_07_restore_vm | Status : SUCCESS ===
    ok
    Test migrate VM ... === TestName: test_08_migrate_vm | Status : SUCCESS ===
    ok
    Test destroy(expunge) Virtual Machine ... === TestName: test_09_expunge_vm | Status : SUCCESS ===
    ok
    Test VPN in VPC ... === TestName: test_vpc_remote_access_vpn | Status : SUCCESS ===
    ok
    Test VPN in VPC ... === TestName: test_vpc_site2site_vpn | Status : SUCCESS ===
    ok
    Test to create service offering ... === TestName: test_01_create_service_offering | Status : SUCCESS ===
    ok
    Test to update existing service offering ... === TestName: test_02_edit_service_offering | Status : SUCCESS ===
    ok
    Test to delete service offering ... === TestName: test_03_delete_service_offering | Status : SUCCESS ===
    ok
    Test create VPC offering ... === TestName: test_01_create_vpc_offering | Status : SUCCESS ===
    ok
    Test VPC offering without load balancing service ... === TestName: test_03_vpc_off_without_lb | Status : EXCEPTION ===
    ERROR
    Test VPC offering without static NAT service ... === TestName: test_04_vpc_off_without_static_nat | Status : EXCEPTION ===
    ERROR
    Test VPC offering without port forwarding service ... === TestName: test_05_vpc_off_without_pf | Status : EXCEPTION ===
    ERROR
    Test VPC offering with invalid services ... === TestName: test_06_vpc_off_invalid_services | Status : SUCCESS ===
    ok
    Test update VPC offering ... === TestName: test_07_update_vpc_off | Status : SUCCESS ===
    ok
    Test list VPC offering ... === TestName: test_08_list_vpc_off | Status : SUCCESS ===
    ok
    test_09_create_redundant_vpc_offering (integration.component.test_vpc_offerings.TestVPCOffering) ... === TestName: test_09_create_redundant_vpc_offering | Status : SUCCESS ===
    ok
    Test start/stop of router after addition of one guest network ... === TestName: test_01_start_stop_router_after_addition_of_one_guest_network | Status : SUCCESS ===
    ok
    Test reboot of router after addition of one guest network ... === TestName: test_02_reboot_router_after_addition_of_one_guest_network | Status : SUCCESS ===
    ok
    Test to change service offering of router after addition of one guest network ... === TestName: test_04_chg_srv_off_router_after_addition_of_one_guest_network | Status : SUCCESS ===
    ok
    Test destroy of router after addition of one guest network ... === TestName: test_05_destroy_router_after_addition_of_one_guest_network | Status : SUCCESS ===
    ok
    Test to stop and start router after creation of VPC ... === TestName: test_01_stop_start_router_after_creating_vpc | Status : SUCCESS ===
    ok
    Test to reboot the router after creating a VPC ... === TestName: test_02_reboot_router_after_creating_vpc | Status : SUCCESS ===
    ok
    Tests to change service offering of the Router after ... === TestName: test_04_change_service_offerring_vpc | Status : SUCCESS ===
    ok
    Test to destroy the router after creating a VPC ... === TestName: test_05_destroy_router_after_creating_vpc | Status : SUCCESS ===
    ok
    
    
    ----------------------------------------------------------------------
    Ran 40 tests in 7144.018s
    
    FAILED (errors=3)
    ```
    The 3 errors are due to cleanups failing. Could be a timing issue, as I could successfully remove the resources manually. It's a known issue, CLOUDSTACK-8935, as mentioned above.
    
    Finally I did a site-to-site VPN test with real hardware and real VMs.
    
    VPCs look like this:
    ![screen shot 2015-10-13 at 14 12 38](https://cloud.githubusercontent.com/assets/1630096/10454583/de87744e-71b5-11e5-854f-2ae54648dae0.png)
    
    ![screen shot 2015-10-13 at 14 12 49](https://cloud.githubusercontent.com/assets/1630096/10454588/e86d870a-71b5-11e5-9bda-4bdb0c1c94d6.png)
    
    Then logging in to each VM, and I could ping the VM in the other VPC over its internal ip:
    
    ![screen shot 2015-10-13 at 14 11 51](https://cloud.githubusercontent.com/assets/1630096/10454598/fcebfba8-71b5-11e5-8b8a-7d52af97b9a1.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---