You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "David Li (Jira)" <ji...@apache.org> on 2020/12/26 16:48:00 UTC
[jira] [Resolved] (ARROW-11032) [C++][FlightRPC] Add benchmark for
local RPC through unix socket
[ https://issues.apache.org/jira/browse/ARROW-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Li resolved ARROW-11032.
------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 9009
[https://github.com/apache/arrow/pull/9009]
> [C++][FlightRPC] Add benchmark for local RPC through unix socket
> ----------------------------------------------------------------
>
> Key: ARROW-11032
> URL: https://issues.apache.org/jira/browse/ARROW-11032
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, FlightRPC
> Reporter: Yibo Cai
> Assignee: Yibo Cai
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For local RPC, unix socket is more efficient than TCP socket through localhost. Benchmark shows it has higher throughput and lower latency. According patch adds "-test_unix" option to enable unix socket based test.
> *One thread (speed 2992 -> 3419, latency 37 -> 35)*
> {code:bash}
> # one thread, TCP socket
> $ ./arrow-flight-benchmark --num-threads 1 --num-perf-runs 100
> Using spawned TCP server
> Server running with pid 13741
> Server host: localhost
> Server port: 31337
> Testing method: DoGet
> Server host: localhost
> Server port: 31337
> Number of perf runs: 100
> Number of concurrent gets/puts: 1
> Batch size: 131040
> Batches read: 976800
> Bytes read: 128000000000
> Nanos: 40786538125
> Speed: 2992.91 MB/s
> Throughput: 23949.1 batches/s
> Latency mean: 38 us
> Latency quantile=0.5: 37 us
> Latency quantile=0.95: 86 us
> Latency quantile=0.99: 446 us
> Latency max: 4747 us
> # one thread, Unix socket
> $ ./arrow-flight-benchmark --num-threads 1 --num-perf-runs 100 --test-unix
> Using spawned Unix server
> Server running with pid 13610
> Server unix socket: /tmp/flight-bench-spawn.sock
> Testing method: DoGet
> Server unix socket: /tmp/flight-bench-spawn.sock
> Number of perf runs: 100
> Number of concurrent gets/puts: 1
> Batch size: 131040
> Batches read: 976800
> Bytes read: 128000000000
> Nanos: 35700637793
> Speed: 3419.28 MB/s
> Throughput: 27360.9 batches/s
> Latency mean: 35 us
> Latency quantile=0.5: 25 us
> Latency quantile=0.95: 104 us
> Latency quantile=0.99: 401 us
> Latency max: 6943 us
> {code}
> *Four threads (speed 9096 -> 10067, latency 49 -> 44)*
> {code:bash}
> # four threads, TCP socket
> $ ./arrow-flight-benchmark --num-threads 4 --num-perf-runs 100
> Using spawned TCP server
> Server running with pid 13860
> Server host: localhost
> Server port: 31337
> Testing method: DoGet
> Server host: localhost
> Server port: 31337
> Number of perf runs: 100
> Number of concurrent gets/puts: 4
> Batch size: 131040
> Batches read: 976800
> Bytes read: 128000000000
> Nanos: 13419015574
> Speed: 9096.82 MB/s
> Throughput: 72792.2 batches/s
> Latency mean: 49 us
> Latency quantile=0.5: 38 us
> Latency quantile=0.95: 69 us
> Latency quantile=0.99: 253 us
> Latency max: 8954 us
> # four threads, Unix socket
> $ ./arrow-flight-benchmark --num-threads 4 --num-perf-runs 100 --test-unix
> Using spawned Unix server
> Server running with pid 14587
> Server unix socket: /tmp/flight-bench-spawn.sock
> Testing method: DoGet
> Server unix socket: /tmp/flight-bench-spawn.sock
> Number of perf runs: 100
> Number of concurrent gets/puts: 4
> Batch size: 131040
> Batches read: 976800
> Bytes read: 128000000000
> Nanos: 12125043167
> Speed: 10067.6 MB/s
> Throughput: 80560.5 batches/s
> Latency mean: 44 us
> Latency quantile=0.5: 5 us
> Latency quantile=0.95: 47 us
> Latency quantile=0.99: 1378 us
> Latency max: 8359 us
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)