[jira] [Created] (ARROW-11032) [C++][Flight] Add benchmark for local RPC through unix socket

["Yibo Cai (Jira)" <ji...@apache.org>]

Yibo Cai created ARROW-11032:
--------------------------------

             Summary: [C++][Flight] Add benchmark for local RPC through unix socket
                 Key: ARROW-11032
                 URL: https://issues.apache.org/jira/browse/ARROW-11032
             Project: Apache Arrow
          Issue Type: Improvement
          Components: FlightRPC
            Reporter: Yibo Cai
            Assignee: Yibo Cai


For local RPC, unix socket is more efficient than TCP socket through localhost. Benchmark shows it has higher throughput and lower latency. According patch adds "-test_unix" option to enable unix socket based test.

*One thread (speed 2992 -> 3419, latency 37 -> 35)*
{code:bash}
# one thread, TCP socket
$ ./arrow-flight-benchmark --num-threads 1 --num-perf-runs 100
Using spawned TCP server
Server running with pid 13741
Server host: localhost
Server port: 31337
Testing method: DoGet
Server host: localhost
Server port: 31337
Number of perf runs: 100
Number of concurrent gets/puts: 1
Batch size: 131040
Batches read: 976800
Bytes read: 128000000000
Nanos: 40786538125
Speed: 2992.91 MB/s
Throughput: 23949.1 batches/s
Latency mean: 38 us
Latency quantile=0.5: 37 us
Latency quantile=0.95: 86 us
Latency quantile=0.99: 446 us
Latency max: 4747 us

# one thread, Unix socket
$ ./arrow-flight-benchmark --num-threads 1 --num-perf-runs 100 --test-unix
Using spawned Unix server
Server running with pid 13610
Server unix socket: /tmp/flight-bench-spawn.sock
Testing method: DoGet
Server unix socket: /tmp/flight-bench-spawn.sock
Number of perf runs: 100
Number of concurrent gets/puts: 1
Batch size: 131040
Batches read: 976800
Bytes read: 128000000000
Nanos: 35700637793
Speed: 3419.28 MB/s
Throughput: 27360.9 batches/s
Latency mean: 35 us
Latency quantile=0.5: 25 us
Latency quantile=0.95: 104 us
Latency quantile=0.99: 401 us
Latency max: 6943 us
{code}
*Four threads (speed 9096 -> 10067, latency 49 -> 44)*
{code:bash}
# four threads, TCP socket
$ ./arrow-flight-benchmark --num-threads 4 --num-perf-runs 100
Using spawned TCP server
Server running with pid 13860
Server host: localhost
Server port: 31337
Testing method: DoGet
Server host: localhost
Server port: 31337
Number of perf runs: 100
Number of concurrent gets/puts: 4
Batch size: 131040
Batches read: 976800
Bytes read: 128000000000
Nanos: 13419015574
Speed: 9096.82 MB/s
Throughput: 72792.2 batches/s
Latency mean: 49 us
Latency quantile=0.5: 38 us
Latency quantile=0.95: 69 us
Latency quantile=0.99: 253 us
Latency max: 8954 us

# four threads, Unix socket
$ ./arrow-flight-benchmark --num-threads 4 --num-perf-runs 100 --test-unix
Using spawned Unix server
Server running with pid 14587
Server unix socket: /tmp/flight-bench-spawn.sock
Testing method: DoGet
Server unix socket: /tmp/flight-bench-spawn.sock
Number of perf runs: 100
Number of concurrent gets/puts: 4
Batch size: 131040
Batches read: 976800
Bytes read: 128000000000
Nanos: 12125043167
Speed: 10067.6 MB/s
Throughput: 80560.5 batches/s
Latency mean: 44 us
Latency quantile=0.5: 5 us
Latency quantile=0.95: 47 us
Latency quantile=0.99: 1378 us
Latency max: 8359 us
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)