You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by "wido (via GitHub)" <gi...@apache.org> on 2024/10/08 19:54:10 UTC
[PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
wido opened a new pull request, #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444
This is a first attempt on getting proper documentation on how to use VXLAN with BGP and EVPN underneath CloudStack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "wido (via GitHub)" <gi...@apache.org>.
wido commented on code in PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1793273872
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
-
-::
-
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
- $ sudo ufw allow proto udp from any to any port 8472
+EVPN Bash script
+~~~~~~~~~~~~~~~~
+The default 'modifyvxlan.sh' script this is installed by CloudStack uses Multicast for VXLAN.
-.. note::
- By default UFW is not enabled on Ubuntu. Executing these commands with the
- firewall disabled does not enable the firewall.
+A different version of this script is available which will use EVPN instead of Multicast and ships with CloudStack by default.
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
+In order to use this script create a symlink on **each** KVM hypervisor
::
- $ sudo service networking restart
- $ sudo reboot
-
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
-
+ $ ln -s /usr/share/modifyvxlan.sh /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan-evpn.sh
+
+This script is also available on `download.cloudstack.org <https://download.cloudstack.org/tools/scripts/vxlan/modifyvxlan.sh>`_.
+
+View the contents of the script to understand it's inner workings, some key items:
+
+- VXLAN (vtep) devices are created using 'nolearning', disabling the use of multicast
+- UDP port 4789 (RFC 7348)
+- IPv4 is used as underlay
+- It assumes an IPv4 (/32) address is configured on the loopback interface and will be the VTEP source
+
+BGP routing daemon
+~~~~~~~~~~~~~~~~~~~
+Using `FRRouting <https://frrouting.org/>`_ as routing daemon is recommended, but not required. In general FRR is a BGP routing daemon with extensive EVPN support.
+
+Refer to the FRRouting documentation on how to install the proper packages and get started with FRR.
+
+A minimal configuration for FRR could look like this:
+
+.. code-block:: bash
+
+ frr defaults traditional
+ hostname hypervisor01
+ log syslog informational
+ no ipv6 forwarding
+ service integrated-vtysh-config
+ !
+ interface ens2f0np0
+ no ipv6 nd suppress-ra
+ !
+ interface ens2f1np1
+ no ipv6 nd suppress-ra
+ !
+ interface lo
+ ip address 10.255.192.12/32
+ ipv6 address 2001:db8:100::1/128
+ !
+ router bgp 4200800212
+ bgp router-id 10.255.192.12
+ no bgp ebgp-requires-policy
+ no bgp default ipv4-unicast
+ no bgp network import-check
+ neighbor uplinks peer-group
+ neighbor uplinks remote-as external
+ neighbor uplinks ebgp-multihop 255
+ neighbor ens2f0np0 interface peer-group uplinks
+ neighbor ens2f1np1 interface peer-group uplinks
+ !
+ address-family ipv4 unicast
+ network 10.255.192.12/32
+ neighbor uplinks activate
+ neighbor uplinks next-hop-self
+ neighbor uplinks soft-reconfiguration inbound
+ neighbor uplinks route-map upstream-v4-in in
+ neighbor uplinks route-map upstream-v4-out out
+ exit-address-family
+ !
+ address-family ipv6 unicast
+ network 2001:db8:100::1/128
+ neighbor uplinks activate
+ neighbor uplinks soft-reconfiguration inbound
+ neighbor uplinks route-map upstream-v6-in in
+ neighbor uplinks route-map upstream-v6-out out
+ exit-address-family
+ !
+ address-family l2vpn evpn
+ neighbor uplinks activate
+ advertise-all-vni
+ advertise-svi-ip
+ exit-address-family
+
+
+This configuration will:
+
+- Establish two BGP sessions using BGP Unnumbered over the two uplinks (ens2f0np0 and ens2f1np1)
+- Twese BGP sessions are usually established with two Top-of-Rack (ToR) switches/routers which are BGP+EVPN capable
Review Comment:
Fixed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "alexandremattioli (via GitHub)" <gi...@apache.org>.
alexandremattioli commented on code in PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1794108901
##########
source/plugins/vxlan.rst:
##########
@@ -87,101 +66,28 @@ have MTU of 1500 bytes, meaning that your physical interface/bridge must have MT
In order to configure "jumbo frames" you can i.e. make physical interface/bridge with 9000 bytes MTU, then all the VXLAN
interfaces will be created with MTU of 8950 bytes, and then MTU size inside Instance can be set to 8950 bytes.
-Important note on max number of multicast groups (and thus VXLAN interfaces)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Default value of "net.ipv4.igmp_max_memberships" (cat /proc/sys/net/ipv4/igmp_max_memberships) is "20", which means that host can be joined to max 20 multicast groups (attach max 20 multicast IPs on the host).
-Since all VXLAN (VTEP) interfaces provisioned on host are multicast-based (belong to certain multicast group, and thus has it's own multicast IP that is used as VTEP), this means that you can not provision more than 20 (working) VXLAN interfaces per host.
-On Linux kernel 3.x you actually can provision more than 20, but ARP request will silently fail and cause client's networking problems
-On Linux kernel 4.x you can NOT provision (start) more than 20 VXLAN interfaces and error message "No buffer space available" can be observed in Cloudstack Agent logs after provisioning required bridges and VXLAN interfaces.
-Increase needed parameter to sane value (i.e. 100 or 200) as required.
-If you need to operate more than 20 Instances from different client's Network, this change above is required.
-
-Advanced: Build kernel and iproute2
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Even if your system doesn't support VXLAN, you can compile the kernel
-and iproute2 by yourself. The following procedure is an example for
-CentOS 6.4.
-
-
-Build kernel
-^^^^^^^^^^^^
-
-::
-
- $ sudo yum groupinstall "Development Tools"
- $ sudo yum install ncurses-devel hmaccalc zlib-devel binutils-devel elfutils-libelf-devel bc
-
- $ KERNEL_VERSION=3.10.4
- # Declare the kernel version you want to build.
-
- $ wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-${KERNEL_VERSION}.tar.xz
- $ tar xvf linux-${KERNEL_VERSION}.tar.xz
- $ cd linux-${KERNEL_VERSION}
- $ cp /boot/config-`uname -r` .config
- $ make oldconfig
- # You may keep hitting enter and choose the default.
-
- $ make menuconfig
- # Dig into "Device Drivers" -> "Network device support",
- # then select "Virtual eXtensible Local Area Network (VXLAN)" and hit space.
- # Make sure it indicates "<M>" (build as module), then Save and Exit.
-
- # You may also want to check "IPv4 NAT" and its child nodes in "IP: Netfilter Configuration"
- # and "IPv6 NAT" and its child nodes in "IPv6: Netfilter Configuration".
- # In 3.10.4, you can find the options in
- # "Networking support" -> "Networking options"
- # -> "Network packet filtering framework (Netfilter)".
-
- $ make # -j N
- # You may use -j N option to make the build process parallel and faster,
- # generally N = 1 + (cores your machine have).
-
- $ sudo make modules_install
- $ sudo make install
- # You would get an error like "ERROR: modinfo: could not find module XXXX" here.
- # This happens mainly due to config structure changes between kernel versions.
- # You can ignore this error, until you find you need the kernel module.
- # If you feel uneasy, you can go back to make menuconfig,
- # find module XXXX by using '/' key, enable the module, build and install the kernel again.
-
- $ sudo vi /etc/grub.conf
- # Make sure the new kernel isn't set as the default and the timeout is long enough,
- # so you can select the new kernel during boot process.
- # It's not a good idea to set the new kernel as the default until you confirm the kernel works fine.
-
- $ sudo reboot
- # Select the new kernel during the boot process.
+In general it's recommend to use an MTU of at least 9000 bytes or larger. Most VXLAN capable network cards and switch support an MTU of up to 9216.
+Using an MTU of 9216 bytes allows for using Jumbo Frames (9000) within guest networks.
-Build iproute2
-^^^^^^^^^^^^^^
-::
-
- $ sudo yum install db4-devel
+VXLAN using Multicast
+---------------------
+The default mode for using VXLAN is Multicast. The required configuration is described below.
- $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git
- $ cd iproute2
- $ git tag
- # Find the version that matches the kernel.
- # If you built kernel 3.10.4 as above, it would be v3.10.0.
+Important note on max number of multicast groups
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- $ git checkout v3.10.0
- $ ./configure
- $ make # -j N
- $ sudo make install
+Default value of "net.ipv4.igmp_max_memberships" (cat /proc/sys/net/ipv4/igmp_max_memberships) is "20", which means that host can be joined to max 20 multicast groups (attach max 20 multicast IPs on the host).
+Since all VXLAN (VTEP) interfaces provisioned on host are multicast-based (belong to certain multicast group, and thus has it's own multicast IP that is used as VTEP), this means that you can not provision more than 20 (working) VXLAN interfaces per host.
-.. note:: Please use rebuild kernel and tools at your own risk.
+Under Linux you can NOT provision (start) more than 20 VXLAN interfaces and error message "No buffer space available" can be observed in Cloudstack Agent logs after provisioning required bridges and VXLAN interfaces.
Review Comment:
```suggestion
Under Linux you can NOT by default provision (start) more than 20 VXLAN interfaces and the error message "No buffer space available" will appear in the Cloudstack Agent logs after provisioning the required bridges and VXLAN interfaces.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "wido (via GitHub)" <gi...@apache.org>.
wido commented on code in PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1793273585
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
-
-::
-
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
- $ sudo ufw allow proto udp from any to any port 8472
+EVPN Bash script
+~~~~~~~~~~~~~~~~
+The default 'modifyvxlan.sh' script this is installed by CloudStack uses Multicast for VXLAN.
-.. note::
- By default UFW is not enabled on Ubuntu. Executing these commands with the
- firewall disabled does not enable the firewall.
+A different version of this script is available which will use EVPN instead of Multicast and ships with CloudStack by default.
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
+In order to use this script create a symlink on **each** KVM hypervisor
::
- $ sudo service networking restart
- $ sudo reboot
-
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
-
+ $ ln -s /usr/share/modifyvxlan.sh /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan-evpn.sh
+
+This script is also available on `download.cloudstack.org <https://download.cloudstack.org/tools/scripts/vxlan/modifyvxlan.sh>`_.
Review Comment:
I have already modified the link so that it will work when 9778 merges
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
-
-::
-
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
- $ sudo ufw allow proto udp from any to any port 8472
+EVPN Bash script
+~~~~~~~~~~~~~~~~
+The default 'modifyvxlan.sh' script this is installed by CloudStack uses Multicast for VXLAN.
-.. note::
- By default UFW is not enabled on Ubuntu. Executing these commands with the
- firewall disabled does not enable the firewall.
+A different version of this script is available which will use EVPN instead of Multicast and ships with CloudStack by default.
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
+In order to use this script create a symlink on **each** KVM hypervisor
::
- $ sudo service networking restart
- $ sudo reboot
-
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
-
+ $ ln -s /usr/share/modifyvxlan.sh /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan-evpn.sh
Review Comment:
Yes! Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "wido (via GitHub)" <gi...@apache.org>.
wido commented on code in PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1793272701
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
-
-::
-
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
- $ sudo ufw allow proto udp from any to any port 8472
+EVPN Bash script
+~~~~~~~~~~~~~~~~
+The default 'modifyvxlan.sh' script this is installed by CloudStack uses Multicast for VXLAN.
-.. note::
- By default UFW is not enabled on Ubuntu. Executing these commands with the
- firewall disabled does not enable the firewall.
+A different version of this script is available which will use EVPN instead of Multicast and ships with CloudStack by default.
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
+In order to use this script create a symlink on **each** KVM hypervisor
::
- $ sudo service networking restart
- $ sudo reboot
-
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
-
+ $ ln -s /usr/share/modifyvxlan.sh /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan-evpn.sh
+
+This script is also available on `download.cloudstack.org <https://download.cloudstack.org/tools/scripts/vxlan/modifyvxlan.sh>`_.
+
+View the contents of the script to understand it's inner workings, some key items:
+
+- VXLAN (vtep) devices are created using 'nolearning', disabling the use of multicast
+- UDP port 4789 (RFC 7348)
Review Comment:
Yes, otherwise UDP packets will not work. 4789 is being used by Cumulus, Arista, Juniper and others as the default VXLAN port.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "rohityadavcloud (via GitHub)" <gi...@apache.org>.
rohityadavcloud commented on PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#issuecomment-2401286550
cc @rajujith @NuxRo @andrijapanicsb @alexandremattioli - can you review the content & technicalities, otherwise LGTM. Thanks for the PR @wido
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "alexandremattioli (via GitHub)" <gi...@apache.org>.
alexandremattioli commented on code in PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1794112856
##########
source/plugins/vxlan.rst:
##########
@@ -17,68 +17,47 @@
The VXLAN Plugin
================
-System Requirements for VXLAN
------------------------------
+General
+-------
+CloudStack supports VXLAN technology to enhance scalability and flexibility in networking designs.
-In CloudStack 4.X.0, this plugin only supports the KVM hypervisor with the
-standard linux bridge.
+Using VXLAN (Virtual Extensible LAN) over traditional VLAN (Virtual LAN) offers several key benefits, especially for modern data centers and cloud networking environments that require high scalability and flexibility.
-The following table lists the requirements for the hypervisor.
+VXLAN overcomes the limitations of traditional VLANs by providing a highly scalable, flexible, and efficient networking solution. It enables the creation of a large number of isolated virtual networks over a common physical infrastructure,
+supports better utilization of network resources through Layer 3 routing capabilities, and simplifies network management and provisioning.
-.. cssclass:: table-striped table-bordered table-hover
+When deploying a VXLAN-based network, there are two options to choose from:
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
-| Item | Requirement | Note |
-+================+===============================================+================================================================================================================+
-| Hypervisor | KVM | OvsVifDriver is not supported by this plugin in CloudStack 4.X, use BridgeVifDriver (default). |
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
-| Linux kernel | version >= 3.7, VXLAN kernel module enabled | It is recommended to use kernel >=3.9, since Linux kernel categorizes the VXLAN driver as experimental <3.9. |
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
-| iproute2 | matches kernel version | |
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
+ • Multicast
+ • EVPN using BGP
-Table: Hypervisor Requirement for VXLAN
-
-
-Linux Distributions that meet the requirements
-----------------------------------------------
-
-The following table lists distributions which meet requirements.
-
-.. cssclass:: table-striped table-bordered table-hover
-
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-| Distribution | Release Version | Kernel Version (Date confirmed) | Note |
-+================+===================+===========================================+================================================================+
-| Ubuntu | 13.04 | 3.8.0 (2013/07/23) | |
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-| Fedora | >= 17 | 3.9.10 (2013/07/23) | Latest kernel packages are available in "update" repository. |
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-| CentOS | >= 6.5 | 2.6.32-431.3.1.el6.x86\_64 (2014/01/21) | |
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-
-Table: List of Linux distributions which meet the hypervisor
-requirements
+While Multicast is the easiest to set up, EVPN offers much more control, scalability, and flexibility. Therefore, it is chosen in most VXLAN network deployments.
Review Comment:
```suggestion
While Multicast is the easiest to set up VXLAN isolation, EVPN offers much more control, scalability, and flexibility. Therefore, it is chosen for most VXLAN network deployments.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "wido (via GitHub)" <gi...@apache.org>.
wido commented on PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#issuecomment-2404136661
> Left some polish suggestions.
>
> Also, the zone creation isolation method screenshot shows VLAN, probably a good idea to replace it with one showing VXLAN to avoid confusion.
>
> Otherwise LGTM. Am actually setting up an env following the doc together with the guys from https://www.netris.io/ @wido
Great! Good to see more people are going to use it. I do recommend to use Frr-10 btw, has some nice bug fixes. But this is frr-stable nowadays.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "wido (via GitHub)" <gi...@apache.org>.
wido merged PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on code in PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1792937353
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
-
-::
-
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
- $ sudo ufw allow proto udp from any to any port 8472
+EVPN Bash script
+~~~~~~~~~~~~~~~~
+The default 'modifyvxlan.sh' script this is installed by CloudStack uses Multicast for VXLAN.
-.. note::
- By default UFW is not enabled on Ubuntu. Executing these commands with the
- firewall disabled does not enable the firewall.
+A different version of this script is available which will use EVPN instead of Multicast and ships with CloudStack by default.
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
+In order to use this script create a symlink on **each** KVM hypervisor
::
- $ sudo service networking restart
- $ sudo reboot
-
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
-
+ $ ln -s /usr/share/modifyvxlan.sh /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan-evpn.sh
+
+This script is also available on `download.cloudstack.org <https://download.cloudstack.org/tools/scripts/vxlan/modifyvxlan.sh>`_.
Review Comment:
once the PR https://github.com/apache/cloudstack/pull/9778 is merged, user can download from the github repository
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
-
-::
-
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
- $ sudo ufw allow proto udp from any to any port 8472
+EVPN Bash script
+~~~~~~~~~~~~~~~~
+The default 'modifyvxlan.sh' script this is installed by CloudStack uses Multicast for VXLAN.
-.. note::
- By default UFW is not enabled on Ubuntu. Executing these commands with the
- firewall disabled does not enable the firewall.
+A different version of this script is available which will use EVPN instead of Multicast and ships with CloudStack by default.
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
+In order to use this script create a symlink on **each** KVM hypervisor
::
- $ sudo service networking restart
- $ sudo reboot
-
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
-
+ $ ln -s /usr/share/modifyvxlan.sh /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan-evpn.sh
+
+This script is also available on `download.cloudstack.org <https://download.cloudstack.org/tools/scripts/vxlan/modifyvxlan.sh>`_.
+
+View the contents of the script to understand it's inner workings, some key items:
+
+- VXLAN (vtep) devices are created using 'nolearning', disabling the use of multicast
+- UDP port 4789 (RFC 7348)
Review Comment:
is it needed to add firewall rules for port 4789 ?
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
-
-::
-
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
- $ sudo ufw allow proto udp from any to any port 8472
+EVPN Bash script
+~~~~~~~~~~~~~~~~
+The default 'modifyvxlan.sh' script this is installed by CloudStack uses Multicast for VXLAN.
-.. note::
- By default UFW is not enabled on Ubuntu. Executing these commands with the
- firewall disabled does not enable the firewall.
+A different version of this script is available which will use EVPN instead of Multicast and ships with CloudStack by default.
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
+In order to use this script create a symlink on **each** KVM hypervisor
::
- $ sudo service networking restart
- $ sudo reboot
-
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
-
+ $ ln -s /usr/share/modifyvxlan.sh /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan-evpn.sh
Review Comment:
should it be
```
$ ln -s /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan-evpn.sh /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan.sh
```
?
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
-
-::
-
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
- $ sudo ufw allow proto udp from any to any port 8472
+EVPN Bash script
+~~~~~~~~~~~~~~~~
+The default 'modifyvxlan.sh' script this is installed by CloudStack uses Multicast for VXLAN.
-.. note::
- By default UFW is not enabled on Ubuntu. Executing these commands with the
- firewall disabled does not enable the firewall.
+A different version of this script is available which will use EVPN instead of Multicast and ships with CloudStack by default.
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
+In order to use this script create a symlink on **each** KVM hypervisor
::
- $ sudo service networking restart
- $ sudo reboot
-
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
-
+ $ ln -s /usr/share/modifyvxlan.sh /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvxlan-evpn.sh
+
+This script is also available on `download.cloudstack.org <https://download.cloudstack.org/tools/scripts/vxlan/modifyvxlan.sh>`_.
+
+View the contents of the script to understand it's inner workings, some key items:
+
+- VXLAN (vtep) devices are created using 'nolearning', disabling the use of multicast
+- UDP port 4789 (RFC 7348)
+- IPv4 is used as underlay
+- It assumes an IPv4 (/32) address is configured on the loopback interface and will be the VTEP source
+
+BGP routing daemon
+~~~~~~~~~~~~~~~~~~~
+Using `FRRouting <https://frrouting.org/>`_ as routing daemon is recommended, but not required. In general FRR is a BGP routing daemon with extensive EVPN support.
+
+Refer to the FRRouting documentation on how to install the proper packages and get started with FRR.
+
+A minimal configuration for FRR could look like this:
+
+.. code-block:: bash
+
+ frr defaults traditional
+ hostname hypervisor01
+ log syslog informational
+ no ipv6 forwarding
+ service integrated-vtysh-config
+ !
+ interface ens2f0np0
+ no ipv6 nd suppress-ra
+ !
+ interface ens2f1np1
+ no ipv6 nd suppress-ra
+ !
+ interface lo
+ ip address 10.255.192.12/32
+ ipv6 address 2001:db8:100::1/128
+ !
+ router bgp 4200800212
+ bgp router-id 10.255.192.12
+ no bgp ebgp-requires-policy
+ no bgp default ipv4-unicast
+ no bgp network import-check
+ neighbor uplinks peer-group
+ neighbor uplinks remote-as external
+ neighbor uplinks ebgp-multihop 255
+ neighbor ens2f0np0 interface peer-group uplinks
+ neighbor ens2f1np1 interface peer-group uplinks
+ !
+ address-family ipv4 unicast
+ network 10.255.192.12/32
+ neighbor uplinks activate
+ neighbor uplinks next-hop-self
+ neighbor uplinks soft-reconfiguration inbound
+ neighbor uplinks route-map upstream-v4-in in
+ neighbor uplinks route-map upstream-v4-out out
+ exit-address-family
+ !
+ address-family ipv6 unicast
+ network 2001:db8:100::1/128
+ neighbor uplinks activate
+ neighbor uplinks soft-reconfiguration inbound
+ neighbor uplinks route-map upstream-v6-in in
+ neighbor uplinks route-map upstream-v6-out out
+ exit-address-family
+ !
+ address-family l2vpn evpn
+ neighbor uplinks activate
+ advertise-all-vni
+ advertise-svi-ip
+ exit-address-family
+
+
+This configuration will:
+
+- Establish two BGP sessions using BGP Unnumbered over the two uplinks (ens2f0np0 and ens2f1np1)
+- Twese BGP sessions are usually established with two Top-of-Rack (ToR) switches/routers which are BGP+EVPN capable
Review Comment:
`Twese` -> `These` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "alexandremattioli (via GitHub)" <gi...@apache.org>.
alexandremattioli commented on code in PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1794118205
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
-
-::
-
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
Review Comment:
@NuxRo Did you truncate the phrase by accident?
##########
source/plugins/vxlan.rst:
##########
@@ -17,68 +17,47 @@
The VXLAN Plugin
================
-System Requirements for VXLAN
------------------------------
+General
+-------
+CloudStack supports VXLAN technology to enhance scalability and flexibility in networking designs.
-In CloudStack 4.X.0, this plugin only supports the KVM hypervisor with the
-standard linux bridge.
+Using VXLAN (Virtual Extensible LAN) over traditional VLAN (Virtual LAN) offers several key benefits, especially for modern data centers and cloud networking environments that require high scalability and flexibility.
-The following table lists the requirements for the hypervisor.
+VXLAN overcomes the limitations of traditional VLANs by providing a highly scalable, flexible, and efficient networking solution. It enables the creation of a large number of isolated virtual networks over a common physical infrastructure,
+supports better utilization of network resources through Layer 3 routing capabilities, and simplifies network management and provisioning.
-.. cssclass:: table-striped table-bordered table-hover
+When deploying a VXLAN-based network, there are two options to choose from:
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
-| Item | Requirement | Note |
-+================+===============================================+================================================================================================================+
-| Hypervisor | KVM | OvsVifDriver is not supported by this plugin in CloudStack 4.X, use BridgeVifDriver (default). |
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
-| Linux kernel | version >= 3.7, VXLAN kernel module enabled | It is recommended to use kernel >=3.9, since Linux kernel categorizes the VXLAN driver as experimental <3.9. |
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
-| iproute2 | matches kernel version | |
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
+ • Multicast
+ • EVPN using BGP
-Table: Hypervisor Requirement for VXLAN
-
-
-Linux Distributions that meet the requirements
-----------------------------------------------
-
-The following table lists distributions which meet requirements.
-
-.. cssclass:: table-striped table-bordered table-hover
-
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-| Distribution | Release Version | Kernel Version (Date confirmed) | Note |
-+================+===================+===========================================+================================================================+
-| Ubuntu | 13.04 | 3.8.0 (2013/07/23) | |
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-| Fedora | >= 17 | 3.9.10 (2013/07/23) | Latest kernel packages are available in "update" repository. |
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-| CentOS | >= 6.5 | 2.6.32-431.3.1.el6.x86\_64 (2014/01/21) | |
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-
-Table: List of Linux distributions which meet the hypervisor
-requirements
+While Multicast is the easiest to set up, EVPN offers much more control, scalability, and flexibility. Therefore, it is chosen in most VXLAN network deployments.
+.. warning::
+ Deploying VXLAN, especially with EVPN, requires extensive networking knowledge which isn't covered by this documentation or CloudStack in general.
+ Make sure to familiarize yourself with VXLAN, BGP and EVPN before attempting to deploy this network technology.
-Check the capability of your system
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+System Requirements / Networking for VXLAN
+------------------------------------------
-To check the capability of your system, execute the following commands.
-::
+The following table lists the requirements for using VXLAN in your deployment:
- $ sudo modprobe vxlan && echo $?
- # Confirm the output is "0".
- # If it's non-0 value or error message, your kernel doesn't have VXLAN kernel module.
- $ ip link add type vxlan help
- # Confirm the output is usage of the command and that it's for VXLAN.
- # If it's not, your iproute2 utility doesn't support VXLAN.
++---------------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
+| Item | Requirement | Note |
++=====================+===============================================+================================================================================================================+
+| Hypervisor | KVM | Only the BridgeVifDriver (default) is supported |
++---------------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
+| Network Card (NIC) | VXLAN offloading | A NIC with VXLAN-offloading support is recommended. For example Mellanox ConnectX-5 or Intel X710 |
++---------------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
+| IP Protocol | IPv4 or IPv6 | CloudStack is agnostic to the IP-protocol being used as underlay. Both IPv4 and IPv6 are supported |
++---------------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
+| MTU | >=1550 | VXLAN has an overhead of 50 bytes, therefor 1550 is the minimum. See the notes below |
Review Comment:
```suggestion
| MTU | >=1550 | VXLAN has an overhead of 50 bytes, therefore 1550 is the minimum. See the notes below |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "alexandremattioli (via GitHub)" <gi...@apache.org>.
alexandremattioli commented on PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#issuecomment-2405560057
> > Left some polish suggestions.
> > Also, the zone creation isolation method screenshot shows VLAN, probably a good idea to replace it with one showing VXLAN to avoid confusion.
> > Otherwise LGTM. Am actually setting up an env following the doc together with the guys from https://www.netris.io/ @wido
>
> Great! Good to see more people are going to use it. I do recommend to use Frr-10 btw, has some nice bug fixes. But this is frr-stable nowadays.
Indeed they are using FRR10. They have a very cool product, lots of interesting networking automation/orchestration, and soon with ACS integration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "alexandremattioli (via GitHub)" <gi...@apache.org>.
alexandremattioli commented on code in PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1794113261
##########
source/plugins/vxlan.rst:
##########
@@ -17,68 +17,47 @@
The VXLAN Plugin
================
-System Requirements for VXLAN
------------------------------
+General
+-------
+CloudStack supports VXLAN technology to enhance scalability and flexibility in networking designs.
-In CloudStack 4.X.0, this plugin only supports the KVM hypervisor with the
-standard linux bridge.
+Using VXLAN (Virtual Extensible LAN) over traditional VLAN (Virtual LAN) offers several key benefits, especially for modern data centers and cloud networking environments that require high scalability and flexibility.
-The following table lists the requirements for the hypervisor.
+VXLAN overcomes the limitations of traditional VLANs by providing a highly scalable, flexible, and efficient networking solution. It enables the creation of a large number of isolated virtual networks over a common physical infrastructure,
+supports better utilization of network resources through Layer 3 routing capabilities, and simplifies network management and provisioning.
-.. cssclass:: table-striped table-bordered table-hover
+When deploying a VXLAN-based network, there are two options to choose from:
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
-| Item | Requirement | Note |
-+================+===============================================+================================================================================================================+
-| Hypervisor | KVM | OvsVifDriver is not supported by this plugin in CloudStack 4.X, use BridgeVifDriver (default). |
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
-| Linux kernel | version >= 3.7, VXLAN kernel module enabled | It is recommended to use kernel >=3.9, since Linux kernel categorizes the VXLAN driver as experimental <3.9. |
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
-| iproute2 | matches kernel version | |
-+----------------+-----------------------------------------------+----------------------------------------------------------------------------------------------------------------+
+ • Multicast
+ • EVPN using BGP
-Table: Hypervisor Requirement for VXLAN
-
-
-Linux Distributions that meet the requirements
-----------------------------------------------
-
-The following table lists distributions which meet requirements.
-
-.. cssclass:: table-striped table-bordered table-hover
-
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-| Distribution | Release Version | Kernel Version (Date confirmed) | Note |
-+================+===================+===========================================+================================================================+
-| Ubuntu | 13.04 | 3.8.0 (2013/07/23) | |
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-| Fedora | >= 17 | 3.9.10 (2013/07/23) | Latest kernel packages are available in "update" repository. |
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-| CentOS | >= 6.5 | 2.6.32-431.3.1.el6.x86\_64 (2014/01/21) | |
-+----------------+-------------------+-------------------------------------------+----------------------------------------------------------------+
-
-Table: List of Linux distributions which meet the hypervisor
-requirements
+While Multicast is the easiest to set up, EVPN offers much more control, scalability, and flexibility. Therefore, it is chosen in most VXLAN network deployments.
+.. warning::
+ Deploying VXLAN, especially with EVPN, requires extensive networking knowledge which isn't covered by this documentation or CloudStack in general.
Review Comment:
```suggestion
Deploying VXLAN, especially with EVPN, requires extensive networking knowledge, which isn't covered by this documentation or CloudStack in general.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "alexandremattioli (via GitHub)" <gi...@apache.org>.
alexandremattioli commented on code in PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1794111683
##########
source/plugins/vxlan.rst:
##########
@@ -17,68 +17,47 @@
The VXLAN Plugin
================
-System Requirements for VXLAN
------------------------------
+General
+-------
+CloudStack supports VXLAN technology to enhance scalability and flexibility in networking designs.
-In CloudStack 4.X.0, this plugin only supports the KVM hypervisor with the
-standard linux bridge.
+Using VXLAN (Virtual Extensible LAN) over traditional VLAN (Virtual LAN) offers several key benefits, especially for modern data centers and cloud networking environments that require high scalability and flexibility.
Review Comment:
```suggestion
Using VXLAN (Virtual Extensible LAN) instead of traditional VLAN (Virtual LAN) for layer 2 isolation method offers several key benefits, especially for modern data centers and cloud networking environments that require high scalability and flexibility.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "NuxRo (via GitHub)" <gi...@apache.org>.
NuxRo commented on code in PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1793657405
##########
source/plugins/vxlan.rst:
##########
@@ -87,101 +66,28 @@ have MTU of 1500 bytes, meaning that your physical interface/bridge must have MT
In order to configure "jumbo frames" you can i.e. make physical interface/bridge with 9000 bytes MTU, then all the VXLAN
interfaces will be created with MTU of 8950 bytes, and then MTU size inside Instance can be set to 8950 bytes.
-Important note on max number of multicast groups (and thus VXLAN interfaces)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Default value of "net.ipv4.igmp_max_memberships" (cat /proc/sys/net/ipv4/igmp_max_memberships) is "20", which means that host can be joined to max 20 multicast groups (attach max 20 multicast IPs on the host).
-Since all VXLAN (VTEP) interfaces provisioned on host are multicast-based (belong to certain multicast group, and thus has it's own multicast IP that is used as VTEP), this means that you can not provision more than 20 (working) VXLAN interfaces per host.
-On Linux kernel 3.x you actually can provision more than 20, but ARP request will silently fail and cause client's networking problems
-On Linux kernel 4.x you can NOT provision (start) more than 20 VXLAN interfaces and error message "No buffer space available" can be observed in Cloudstack Agent logs after provisioning required bridges and VXLAN interfaces.
-Increase needed parameter to sane value (i.e. 100 or 200) as required.
-If you need to operate more than 20 Instances from different client's Network, this change above is required.
-
-Advanced: Build kernel and iproute2
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Even if your system doesn't support VXLAN, you can compile the kernel
-and iproute2 by yourself. The following procedure is an example for
-CentOS 6.4.
-
-
-Build kernel
-^^^^^^^^^^^^
-
-::
-
- $ sudo yum groupinstall "Development Tools"
- $ sudo yum install ncurses-devel hmaccalc zlib-devel binutils-devel elfutils-libelf-devel bc
-
- $ KERNEL_VERSION=3.10.4
- # Declare the kernel version you want to build.
-
- $ wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-${KERNEL_VERSION}.tar.xz
- $ tar xvf linux-${KERNEL_VERSION}.tar.xz
- $ cd linux-${KERNEL_VERSION}
- $ cp /boot/config-`uname -r` .config
- $ make oldconfig
- # You may keep hitting enter and choose the default.
-
- $ make menuconfig
- # Dig into "Device Drivers" -> "Network device support",
- # then select "Virtual eXtensible Local Area Network (VXLAN)" and hit space.
- # Make sure it indicates "<M>" (build as module), then Save and Exit.
-
- # You may also want to check "IPv4 NAT" and its child nodes in "IP: Netfilter Configuration"
- # and "IPv6 NAT" and its child nodes in "IPv6: Netfilter Configuration".
- # In 3.10.4, you can find the options in
- # "Networking support" -> "Networking options"
- # -> "Network packet filtering framework (Netfilter)".
-
- $ make # -j N
- # You may use -j N option to make the build process parallel and faster,
- # generally N = 1 + (cores your machine have).
-
- $ sudo make modules_install
- $ sudo make install
- # You would get an error like "ERROR: modinfo: could not find module XXXX" here.
- # This happens mainly due to config structure changes between kernel versions.
- # You can ignore this error, until you find you need the kernel module.
- # If you feel uneasy, you can go back to make menuconfig,
- # find module XXXX by using '/' key, enable the module, build and install the kernel again.
-
- $ sudo vi /etc/grub.conf
- # Make sure the new kernel isn't set as the default and the timeout is long enough,
- # so you can select the new kernel during boot process.
- # It's not a good idea to set the new kernel as the default until you confirm the kernel works fine.
-
- $ sudo reboot
- # Select the new kernel during the boot process.
+In general it's recommend to use an MTU of at least 9000 bytes or larger. Most VXLAN capable network cards and switch support an MTU of up to 9216.
Review Comment:
```suggestion
In general it's recommended to use an MTU of at least 9000 bytes or larger. Most VXLAN capable network cards and switch support an MTU of up to 9216.
```
##########
source/plugins/vxlan.rst:
##########
@@ -87,101 +66,28 @@ have MTU of 1500 bytes, meaning that your physical interface/bridge must have MT
In order to configure "jumbo frames" you can i.e. make physical interface/bridge with 9000 bytes MTU, then all the VXLAN
interfaces will be created with MTU of 8950 bytes, and then MTU size inside Instance can be set to 8950 bytes.
-Important note on max number of multicast groups (and thus VXLAN interfaces)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Default value of "net.ipv4.igmp_max_memberships" (cat /proc/sys/net/ipv4/igmp_max_memberships) is "20", which means that host can be joined to max 20 multicast groups (attach max 20 multicast IPs on the host).
-Since all VXLAN (VTEP) interfaces provisioned on host are multicast-based (belong to certain multicast group, and thus has it's own multicast IP that is used as VTEP), this means that you can not provision more than 20 (working) VXLAN interfaces per host.
-On Linux kernel 3.x you actually can provision more than 20, but ARP request will silently fail and cause client's networking problems
-On Linux kernel 4.x you can NOT provision (start) more than 20 VXLAN interfaces and error message "No buffer space available" can be observed in Cloudstack Agent logs after provisioning required bridges and VXLAN interfaces.
-Increase needed parameter to sane value (i.e. 100 or 200) as required.
-If you need to operate more than 20 Instances from different client's Network, this change above is required.
-
-Advanced: Build kernel and iproute2
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Even if your system doesn't support VXLAN, you can compile the kernel
-and iproute2 by yourself. The following procedure is an example for
-CentOS 6.4.
-
-
-Build kernel
-^^^^^^^^^^^^
-
-::
-
- $ sudo yum groupinstall "Development Tools"
- $ sudo yum install ncurses-devel hmaccalc zlib-devel binutils-devel elfutils-libelf-devel bc
-
- $ KERNEL_VERSION=3.10.4
- # Declare the kernel version you want to build.
-
- $ wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-${KERNEL_VERSION}.tar.xz
- $ tar xvf linux-${KERNEL_VERSION}.tar.xz
- $ cd linux-${KERNEL_VERSION}
- $ cp /boot/config-`uname -r` .config
- $ make oldconfig
- # You may keep hitting enter and choose the default.
-
- $ make menuconfig
- # Dig into "Device Drivers" -> "Network device support",
- # then select "Virtual eXtensible Local Area Network (VXLAN)" and hit space.
- # Make sure it indicates "<M>" (build as module), then Save and Exit.
-
- # You may also want to check "IPv4 NAT" and its child nodes in "IP: Netfilter Configuration"
- # and "IPv6 NAT" and its child nodes in "IPv6: Netfilter Configuration".
- # In 3.10.4, you can find the options in
- # "Networking support" -> "Networking options"
- # -> "Network packet filtering framework (Netfilter)".
-
- $ make # -j N
- # You may use -j N option to make the build process parallel and faster,
- # generally N = 1 + (cores your machine have).
-
- $ sudo make modules_install
- $ sudo make install
- # You would get an error like "ERROR: modinfo: could not find module XXXX" here.
- # This happens mainly due to config structure changes between kernel versions.
- # You can ignore this error, until you find you need the kernel module.
- # If you feel uneasy, you can go back to make menuconfig,
- # find module XXXX by using '/' key, enable the module, build and install the kernel again.
-
- $ sudo vi /etc/grub.conf
- # Make sure the new kernel isn't set as the default and the timeout is long enough,
- # so you can select the new kernel during boot process.
- # It's not a good idea to set the new kernel as the default until you confirm the kernel works fine.
-
- $ sudo reboot
- # Select the new kernel during the boot process.
+In general it's recommend to use an MTU of at least 9000 bytes or larger. Most VXLAN capable network cards and switch support an MTU of up to 9216.
+Using an MTU of 9216 bytes allows for using Jumbo Frames (9000) within guest networks.
-Build iproute2
-^^^^^^^^^^^^^^
-::
-
- $ sudo yum install db4-devel
+VXLAN using Multicast
+---------------------
+The default mode for using VXLAN is Multicast. The required configuration is described below.
- $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git
- $ cd iproute2
- $ git tag
- # Find the version that matches the kernel.
- # If you built kernel 3.10.4 as above, it would be v3.10.0.
+Important note on max number of multicast groups
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- $ git checkout v3.10.0
- $ ./configure
- $ make # -j N
- $ sudo make install
+Default value of "net.ipv4.igmp_max_memberships" (cat /proc/sys/net/ipv4/igmp_max_memberships) is "20", which means that host can be joined to max 20 multicast groups (attach max 20 multicast IPs on the host).
+Since all VXLAN (VTEP) interfaces provisioned on host are multicast-based (belong to certain multicast group, and thus has it's own multicast IP that is used as VTEP), this means that you can not provision more than 20 (working) VXLAN interfaces per host.
-.. note:: Please use rebuild kernel and tools at your own risk.
+Under Linux you can NOT provision (start) more than 20 VXLAN interfaces and error message "No buffer space available" can be observed in Cloudstack Agent logs after provisioning required bridges and VXLAN interfaces.
+Increase needed parameter to sane value (i.e. 100 or 200) as required.
Review Comment:
```suggestion
Increase the needed parameter to an appropriate value (i.e. 100 or 200) as required.
```
##########
source/plugins/vxlan.rst:
##########
@@ -87,101 +66,28 @@ have MTU of 1500 bytes, meaning that your physical interface/bridge must have MT
In order to configure "jumbo frames" you can i.e. make physical interface/bridge with 9000 bytes MTU, then all the VXLAN
interfaces will be created with MTU of 8950 bytes, and then MTU size inside Instance can be set to 8950 bytes.
-Important note on max number of multicast groups (and thus VXLAN interfaces)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Default value of "net.ipv4.igmp_max_memberships" (cat /proc/sys/net/ipv4/igmp_max_memberships) is "20", which means that host can be joined to max 20 multicast groups (attach max 20 multicast IPs on the host).
-Since all VXLAN (VTEP) interfaces provisioned on host are multicast-based (belong to certain multicast group, and thus has it's own multicast IP that is used as VTEP), this means that you can not provision more than 20 (working) VXLAN interfaces per host.
-On Linux kernel 3.x you actually can provision more than 20, but ARP request will silently fail and cause client's networking problems
-On Linux kernel 4.x you can NOT provision (start) more than 20 VXLAN interfaces and error message "No buffer space available" can be observed in Cloudstack Agent logs after provisioning required bridges and VXLAN interfaces.
-Increase needed parameter to sane value (i.e. 100 or 200) as required.
-If you need to operate more than 20 Instances from different client's Network, this change above is required.
-
-Advanced: Build kernel and iproute2
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Even if your system doesn't support VXLAN, you can compile the kernel
-and iproute2 by yourself. The following procedure is an example for
-CentOS 6.4.
-
-
-Build kernel
-^^^^^^^^^^^^
-
-::
-
- $ sudo yum groupinstall "Development Tools"
- $ sudo yum install ncurses-devel hmaccalc zlib-devel binutils-devel elfutils-libelf-devel bc
-
- $ KERNEL_VERSION=3.10.4
- # Declare the kernel version you want to build.
-
- $ wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-${KERNEL_VERSION}.tar.xz
- $ tar xvf linux-${KERNEL_VERSION}.tar.xz
- $ cd linux-${KERNEL_VERSION}
- $ cp /boot/config-`uname -r` .config
- $ make oldconfig
- # You may keep hitting enter and choose the default.
-
- $ make menuconfig
- # Dig into "Device Drivers" -> "Network device support",
- # then select "Virtual eXtensible Local Area Network (VXLAN)" and hit space.
- # Make sure it indicates "<M>" (build as module), then Save and Exit.
-
- # You may also want to check "IPv4 NAT" and its child nodes in "IP: Netfilter Configuration"
- # and "IPv6 NAT" and its child nodes in "IPv6: Netfilter Configuration".
- # In 3.10.4, you can find the options in
- # "Networking support" -> "Networking options"
- # -> "Network packet filtering framework (Netfilter)".
-
- $ make # -j N
- # You may use -j N option to make the build process parallel and faster,
- # generally N = 1 + (cores your machine have).
-
- $ sudo make modules_install
- $ sudo make install
- # You would get an error like "ERROR: modinfo: could not find module XXXX" here.
- # This happens mainly due to config structure changes between kernel versions.
- # You can ignore this error, until you find you need the kernel module.
- # If you feel uneasy, you can go back to make menuconfig,
- # find module XXXX by using '/' key, enable the module, build and install the kernel again.
-
- $ sudo vi /etc/grub.conf
- # Make sure the new kernel isn't set as the default and the timeout is long enough,
- # so you can select the new kernel during boot process.
- # It's not a good idea to set the new kernel as the default until you confirm the kernel works fine.
-
- $ sudo reboot
- # Select the new kernel during the boot process.
+In general it's recommend to use an MTU of at least 9000 bytes or larger. Most VXLAN capable network cards and switch support an MTU of up to 9216.
+Using an MTU of 9216 bytes allows for using Jumbo Frames (9000) within guest networks.
-Build iproute2
-^^^^^^^^^^^^^^
-::
-
- $ sudo yum install db4-devel
+VXLAN using Multicast
+---------------------
+The default mode for using VXLAN is Multicast. The required configuration is described below.
- $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git
- $ cd iproute2
- $ git tag
- # Find the version that matches the kernel.
- # If you built kernel 3.10.4 as above, it would be v3.10.0.
+Important note on max number of multicast groups
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- $ git checkout v3.10.0
- $ ./configure
- $ make # -j N
- $ sudo make install
+Default value of "net.ipv4.igmp_max_memberships" (cat /proc/sys/net/ipv4/igmp_max_memberships) is "20", which means that host can be joined to max 20 multicast groups (attach max 20 multicast IPs on the host).
+Since all VXLAN (VTEP) interfaces provisioned on host are multicast-based (belong to certain multicast group, and thus has it's own multicast IP that is used as VTEP), this means that you can not provision more than 20 (working) VXLAN interfaces per host.
-.. note:: Please use rebuild kernel and tools at your own risk.
+Under Linux you can NOT provision (start) more than 20 VXLAN interfaces and error message "No buffer space available" can be observed in Cloudstack Agent logs after provisioning required bridges and VXLAN interfaces.
+Increase needed parameter to sane value (i.e. 100 or 200) as required.
-Configure CloudStack to use VXLAN Plugin
--------------------------------------
+If you need to operate more than 20 Instances from different client's Network, this change above is required.
Review Comment:
```suggestion
If you need to operate more than 20 Instances from different client networks, the change above is required.
```
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
-
-::
-
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
- $ sudo ufw allow proto udp from any to any port 8472
+EVPN Bash script
+~~~~~~~~~~~~~~~~
+The default 'modifyvxlan.sh' script this is installed by CloudStack uses Multicast for VXLAN.
-.. note::
- By default UFW is not enabled on Ubuntu. Executing these commands with the
- firewall disabled does not enable the firewall.
+A different version of this script is available which will use EVPN instead of Multicast and ships with CloudStack by default.
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
+In order to use this script create a symlink on **each** KVM hypervisor
::
-
- $ sudo service networking restart
- $ sudo reboot
-
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
-
+ $ cd /usr/share
+ $ ln -s cloudstack-common/scripts/vm/network/vnet/modifyvxlan-evpn.sh modifyvxlan.sh
+
+This script is also available in the CloudStack `GIT repository <https://raw.githubusercontent.com/apache/cloudstack/refs/heads/main/scripts/vm/network/vnet/modifyvxlan-evpn.sh>`_.
+
+View the contents of the script to understand it's inner workings, some key items:
Review Comment:
```suggestion
View the contents of the script to understand its inner workings, some key items:
```
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
-
-::
-
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
Review Comment:
```suggestion
- The modified script is required
```
##########
source/plugins/vxlan.rst:
##########
@@ -328,70 +234,134 @@ Since VXLAN uses UDP packet to forward encapsulated the L2 frames,
UDP/8472 port must be opened.
-Configure in RHEL or CentOS
-'''''''''''''''''''''''''''
-
-RHEL and CentOS use iptables for firewalling the system, you can open
-extra ports by executing the following iptable commands:
+Make sure that your firewall (firewalld, ufw, ...) allows UDP packets on port 8472, as an example:
::
- $ sudo iptables -I INPUT -p udp -m udp --dport 8472 -j ACCEPT
-
-
-These iptable settings are not persistent accross reboots, we have to
-save them first.
-
-::
-
- $ sudo iptables-save > /etc/sysconfig/iptables
-
-
-With this configuration you should be able to restart the Network,
-although a reboot is recommended to see if everything works properly.
-
-::
-
- $ sudo service network restart
- $ sudo reboot
+ $ sudo firewall-cmd --zone=public --permanent --add-port=8472/udp
+ $ sudo ufw allow proto udp from any to any port 8472
-.. warning::
- Make sure you have an alternative way like IPMI or ILO to reach the machine
- in case you made a configuration error and the Network stops functioning!
+VXLAN using EVPN
+---------------------
+Using VXLAN with BGP+EVPN as underlay is more complex to set up, but does allow for more scaling and provides much more flexibility.
-Configure in Ubuntu
-'''''''''''''''''''
+This documentation can not cover all elements of deploying BGP+EVPN in your environment.
-The default firewall under Ubuntu is UFW (Uncomplicated FireWall), which
-is a Python wrapper around iptables.
+It is recommend to read `this blogpost <https://vincent.bernat.ch/en/blog/2017-vxlan-bgp-evpn>`_ before you continue.
-To open the required ports, execute the following commands:
+The main items for using EVPN:
-::
+- BGP Routing Daemon on the hypervisor
+- No LACP/Bonding will be used
+- Modified script is required to use EVPN instead of Multicast
- $ sudo ufw allow proto udp from any to any port 8472
+EVPN Bash script
+~~~~~~~~~~~~~~~~
+The default 'modifyvxlan.sh' script this is installed by CloudStack uses Multicast for VXLAN.
Review Comment:
```suggestion
The default 'modifyvxlan.sh' script installed by CloudStack uses Multicast for VXLAN.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] vxlan: Update VXLAN documentation to include EVPN [cloudstack-documentation]
Posted by "wido (via GitHub)" <gi...@apache.org>.
wido commented on PR #444:
URL: https://github.com/apache/cloudstack-documentation/pull/444#issuecomment-2415955433
All changes approved, all checks passed. I see no objections, merging this one soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org