[GitHub] [cloudstack] richardlawley opened a new issue #3353: 4.11.2 HA VRs with Second Public NIC get different routes between Startup and Failover

[GitBox <gi...@apache.org>]

richardlawley opened a new issue #3353: 4.11.2 HA VRs with Second Public NIC get different routes between Startup and Failover
URL: https://github.com/apache/cloudstack/issues/3353
 
 
   <!--
   Verify first that your issue/request is not already reported on GitHub.
   Also test if the latest release and master branch are affected too.
   Always add information AFTER of these HTML comments, but no need to delete the comments.
   -->
   
   ##### ISSUE TYPE
   <!-- Pick one below and delete the rest -->
    * Bug Report
   
   ##### COMPONENT NAME
   <!--
   Categorize the issue, e.g. API, VR, VPN, UI, etc.
   -->
   ~~~
   VR
   ~~~
   
   ##### CLOUDSTACK VERSION
   <!--
   New line separated list of affected versions, commit ID for issues on master branch.
   -->
   
   ~~~
   4.11.2
   ~~~
   
   ##### CONFIGURATION
   <!--
   Information about the configuration if relevant, e.g. basic network, advanced networking, etc.  N/A otherwise
   -->
   Advanced Networking, HA VRs
   
   ##### OS / ENVIRONMENT
   <!--
   Information about the environment if relevant, N/A otherwise
   -->
   XenServer 6.5
   
   ##### SUMMARY
   <!-- Explain the problem/feature briefly -->
   When a network has public IPs from multiple ranges and a VM has a Static NAT rule from the second IP range (different range from the Source NAT IP), the outbound IP changes when a VR fails over.  This is because default routes for each interface are not restored when a failover occurs, but are added correctly when the VR is built.
   
   ##### STEPS TO REPRODUCE
   <!--
   For bugs, show exactly how to reproduce the problem, using a minimal test-case. Use Screenshots if accurate.
   
   For new features, show how the feature would be used.
   -->
   
   1. Create zone with 2 or more public IP ranges
   2. Create an advanced network, allocate IPs until you get IPs from both networks (or use the database to assign a specific IP)
   3. Create a VM and assign a Static NAT IP to the IP from the second range
   4. Reboot both VRs at the same time to get a "clean" situation
   5. Log into Master VR
   6. Run `ip route table list all > /root/route.1`
   7. Run `systemctl restart keepalived` to force a failover without rebuilding VR
   8. Log into the new VR and fail back using `systemctl restart keepalived`
   9. Run `ip route table list all > /root/route.2`
   10. Compare routing tables with diff /root/route.1 /root/route.2
   
   
   
   <!-- You can also paste gist.github.com links for larger files -->
   
   ##### EXPECTED RESULTS
   <!-- What did you expect to happen when running the steps above? -->
   
   ~~~
   Routing table should be identical between first build and failover
   ~~~
   
   ##### ACTUAL RESULTS
   <!-- What actually happened? -->
   Default routes for each interface were missing, resulting in the outgoing traffic from the VM with static NAT originating from the Source NAT IP
   
   <!-- Paste verbatim command output between quotes below -->
   ~~~
   # diff /root/route.1 /root/route.2
   2d1
   < default via X.X.X.X dev eth2 table Table_eth2 proto static 
   6d4
   < default via X.X.X.X dev eth3 table Table_eth3 proto static
   ~~~
   
   We are able to fix this manually by re-adding the route:
   ~~~
   ip route add default via X.X.X.X table Table_eth3 proto static
   ~~~
   NB - the eth2 route doesn't seem to be necessary, but adding the eth3 route fixes the originating IPs.  

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services