You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Bharat Kumar <bh...@citrix.com> on 2013/05/24 12:52:17 UTC
Regarding the RPS and RFS support in the virtual router.
Hi All,
I have enabled the RPS and RFS support in the router vm and performed some tests, but cloud not see any improvements. If any one know about RPS (receive packet steering ) and RFS (receive flow steering ) please comment. I am not sure if the RPS and RFS is working.
The router has 4 cores and 4GB of ram.
command used to enable rps
echo f > /sys/class/net/eth0/queues/rx-0/rps_cpus
commands used to enable rfs
echo 1024 > /proc/sys/net/core/rps_sock_flow_entries
echo 1024 > /sys/class/net/eth0/queues/rx-0/rps_flow_c
kernel details
Linux 3.2.0-4-686-pae #1 SMP Debian 3.2.35-2 i686 GNU/Linux
The following are the test results
Test Without rps
------------------------------------------------------
Testing with the following command line:
/usr/bin/netperf -l 60 -H 10.147.28.149 -c -C -t TCP_RR -i 700,3 -I 99,5 -- -r 1,1 -s 0 -S 0
TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.147.28.149 (10.147.28.149) port 0 AF_INET : +/-2.500% @ 99% conf.
Local /Remote
Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
Send Recv Size Size Time Rate local remote local remote
bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
2048 256 1 1 60.00 4280.98 16.30 4.06 38.081 37.977
2048 2240
Test With RPS
------------------------------------------------------
usr/bin/netperf -l 60 -H 10.147.28.149 -c -C -t TCP_RR -i 700,3 -I 99,5 -- -r 1,1 -s 0 -S 0
TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.147.28.149 (10.147.28.149) port 0 AF_INET : +/-2.500% @ 99% conf.
Local /Remote
Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
Send Recv Size Size Time Rate local remote local remote
bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
2048 256 1 1 60.00 3926.40 14.47 3.68 37.093 37.561
2048 2240
Test with RPS and RFS
------------------------------------------------------
/usr/bin/netperf -l 60 -H 10.147.28.149 -c -C -t TCP_RR -i 700,3 -I 99,5 -- -r 1,1 -s 0 -S 0
TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.147.28.149 (10.147.28.149) port 0 AF_INET : +/-2.500% @ 99% conf.
Local /Remote
Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
Send Recv Size Size Time Rate local remote local remote
bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
2048 256 1 1 60.00 3903.11 14.74 3.76 37.801 38.494
2048 2240
Re: Regarding the RPS and RFS support in the virtual router.
Posted by Bharat Kumar <bh...@citrix.com>.
Thank you for the input Chiradeep.
On May 28, 2013, at 11:13 PM, Chiradeep Vittal <Ch...@citrix.com> wrote:
> The test is invalid -- it is only testing one core.
> RPS steers the packet processing to a CPU core based on the hash of the
> TCP/UDP 4-tuple in the header (src addr, dest addr, src port dst port).
> With a single-stream test, the hash will send every packet to the same cpu
>
> RFS is similarly based on the hash of the 4-tuple. The difference is that
> the packets are steered to the CPU of the application thread that is
> handling the flow.
>
> Here's the guide on how to measure aggregate throughput:
> http://s.apache.org/EID
>
>
> On 5/24/13 3:52 AM, "Bharat Kumar" <bh...@citrix.com> wrote:
>
>> Hi All,
>>
>> I have enabled the RPS and RFS support in the router vm and performed
>> some tests, but cloud not see any improvements. If any one know about
>> RPS (receive packet steering ) and RFS (receive flow steering ) please
>> comment. I am not sure if the RPS and RFS is working.
>>
>> The router has 4 cores and 4GB of ram.
>> command used to enable rps
>> echo f > /sys/class/net/eth0/queues/rx-0/rps_cpus
>> commands used to enable rfs
>> echo 1024 > /proc/sys/net/core/rps_sock_flow_entries
>> echo 1024 > /sys/class/net/eth0/queues/rx-0/rps_flow_c
>>
>> kernel details
>> Linux 3.2.0-4-686-pae #1 SMP Debian 3.2.35-2 i686 GNU/Linux
>>
>> The following are the test results
>> Test Without rps
>> ------------------------------------------------------
>> Testing with the following command line:
>> /usr/bin/netperf -l 60 -H 10.147.28.149 -c -C -t TCP_RR -i 700,3 -I
>> 99,5 -- -r 1,1 -s 0 -S 0
>>
>> TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
>> 10.147.28.149 (10.147.28.149) port 0 AF_INET : +/-2.500% @ 99% conf.
>> Local /Remote
>> Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
>> Send Recv Size Size Time Rate local remote local remote
>> bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
>>
>> 2048 256 1 1 60.00 4280.98 16.30 4.06 38.081 37.977
>> 2048 2240
>>
>> Test With RPS
>> ------------------------------------------------------
>> usr/bin/netperf -l 60 -H 10.147.28.149 -c -C -t TCP_RR -i 700,3 -I
>> 99,5 -- -r 1,1 -s 0 -S 0
>> TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
>> 10.147.28.149 (10.147.28.149) port 0 AF_INET : +/-2.500% @ 99% conf.
>> Local /Remote
>> Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
>> Send Recv Size Size Time Rate local remote local remote
>> bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
>>
>> 2048 256 1 1 60.00 3926.40 14.47 3.68 37.093 37.561
>> 2048 2240
>>
>> Test with RPS and RFS
>> ------------------------------------------------------
>> /usr/bin/netperf -l 60 -H 10.147.28.149 -c -C -t TCP_RR -i 700,3 -I
>> 99,5 -- -r 1,1 -s 0 -S 0
>> TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
>> 10.147.28.149 (10.147.28.149) port 0 AF_INET : +/-2.500% @ 99% conf.
>> Local /Remote
>> Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
>> Send Recv Size Size Time Rate local remote local remote
>> bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
>>
>> 2048 256 1 1 60.00 3903.11 14.74 3.76 37.801 38.494
>> 2048 2240
>>
>>
>>
>>
>
Re: Regarding the RPS and RFS support in the virtual router.
Posted by Chiradeep Vittal <Ch...@citrix.com>.
The test is invalid -- it is only testing one core.
RPS steers the packet processing to a CPU core based on the hash of the
TCP/UDP 4-tuple in the header (src addr, dest addr, src port dst port).
With a single-stream test, the hash will send every packet to the same cpu
RFS is similarly based on the hash of the 4-tuple. The difference is that
the packets are steered to the CPU of the application thread that is
handling the flow.
Here's the guide on how to measure aggregate throughput:
http://s.apache.org/EID
On 5/24/13 3:52 AM, "Bharat Kumar" <bh...@citrix.com> wrote:
>Hi All,
>
>I have enabled the RPS and RFS support in the router vm and performed
>some tests, but cloud not see any improvements. If any one know about
>RPS (receive packet steering ) and RFS (receive flow steering ) please
>comment. I am not sure if the RPS and RFS is working.
>
>The router has 4 cores and 4GB of ram.
>command used to enable rps
>echo f > /sys/class/net/eth0/queues/rx-0/rps_cpus
>commands used to enable rfs
>echo 1024 > /proc/sys/net/core/rps_sock_flow_entries
>echo 1024 > /sys/class/net/eth0/queues/rx-0/rps_flow_c
>
>kernel details
>Linux 3.2.0-4-686-pae #1 SMP Debian 3.2.35-2 i686 GNU/Linux
>
>The following are the test results
>Test Without rps
>------------------------------------------------------
>Testing with the following command line:
>/usr/bin/netperf -l 60 -H 10.147.28.149 -c -C -t TCP_RR -i 700,3 -I
>99,5 -- -r 1,1 -s 0 -S 0
>
>TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
>10.147.28.149 (10.147.28.149) port 0 AF_INET : +/-2.500% @ 99% conf.
>Local /Remote
>Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
>Send Recv Size Size Time Rate local remote local remote
>bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
>
>2048 256 1 1 60.00 4280.98 16.30 4.06 38.081 37.977
>2048 2240
>
>Test With RPS
>------------------------------------------------------
>usr/bin/netperf -l 60 -H 10.147.28.149 -c -C -t TCP_RR -i 700,3 -I
>99,5 -- -r 1,1 -s 0 -S 0
>TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
>10.147.28.149 (10.147.28.149) port 0 AF_INET : +/-2.500% @ 99% conf.
>Local /Remote
>Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
>Send Recv Size Size Time Rate local remote local remote
>bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
>
>2048 256 1 1 60.00 3926.40 14.47 3.68 37.093 37.561
>2048 2240
>
>Test with RPS and RFS
>------------------------------------------------------
>/usr/bin/netperf -l 60 -H 10.147.28.149 -c -C -t TCP_RR -i 700,3 -I
>99,5 -- -r 1,1 -s 0 -S 0
>TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
>10.147.28.149 (10.147.28.149) port 0 AF_INET : +/-2.500% @ 99% conf.
>Local /Remote
>Socket Size Request Resp. Elapsed Trans. CPU CPU S.dem S.dem
>Send Recv Size Size Time Rate local remote local remote
>bytes bytes bytes bytes secs. per sec % S % S us/Tr us/Tr
>
>2048 256 1 1 60.00 3903.11 14.74 3.76 37.801 38.494
>2048 2240
>
>
>
>