You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/04 14:26:44 UTC
[GitHub] [arrow] lidavidm opened a new pull request #10242: ARROW-12648: [C++][FlightRPC] Enable TLS for Flight benchmark
lidavidm opened a new pull request #10242:
URL: https://github.com/apache/arrow/pull/10242
This allows you to enable TLS with a self-signed certificate in the Flight benchmark. Quick testing shows about a 20% performance impact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] cyb70289 closed pull request #10242: ARROW-12648: [C++][FlightRPC] Enable TLS for Flight benchmark
Posted by GitBox <gi...@apache.org>.
cyb70289 closed pull request #10242:
URL: https://github.com/apache/arrow/pull/10242
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #10242: ARROW-12648: [C++][FlightRPC] Enable TLS for Flight benchmark
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #10242:
URL: https://github.com/apache/arrow/pull/10242#issuecomment-831984631
https://issues.apache.org/jira/browse/ARROW-12648
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm edited a comment on pull request #10242: ARROW-12648: [C++][FlightRPC] Enable TLS for Flight benchmark
Posted by GitBox <gi...@apache.org>.
lidavidm edited a comment on pull request #10242:
URL: https://github.com/apache/arrow/pull/10242#issuecomment-831985814
I tested this between two AWS EC2 instances (t3.xlarge) using a self-signed elliptic-curve certificate and one month's worth of the NYC taxi dataset. The performance impact is about 20%.
Generate key:
```
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 -x509 -nodes -days 365 -out cert.pem -keyout key.pem
```
Plaintext:
```
$ arrow-flight-benchmark -server_host ip-172-31-73-63.ec2.internal -test_put -num_perf_runs=4 -num_streams=4 -num_threads=1 -data_file ~/data.feather
Testing method: DoPut
Using standalone TCP server
Server host: ip-172-31-73-63.ec2.internal
Server port: 31337
Number of perf runs: 4
Number of concurrent gets/puts: 1
Batch size: 5265782
Batches written: 3456
Bytes written: 18198543232
Nanos: 36934869371
Speed: 469.894 MB/s
Throughput: 93.5701 batches/s
Latency mean: 8501 us
Latency quantile=0.5: 8676 us
Latency quantile=0.95: 9189 us
Latency quantile=0.99: 10791 us
Latency max: 20421 us
```
TLS:
```
$ arrow-flight-benchmark -server_host ip-172-31-73-63.ec2.internal -test_put -num_perf_runs=4 -num_streams=4 -num_threads=1 -data_file ~/data.feather -cert_file cert.pem
Testing method: DoPut
Using standalone TCP server
Server host: ip-172-31-73-63.ec2.internal
Server port: 31337
Number of perf runs: 4
Number of concurrent gets/puts: 1
Batch size: 5265782
Batches written: 3456
Bytes written: 18198543232
Nanos: 43250718195
Speed: 401.276 MB/s
Throughput: 79.9062 batches/s
Latency mean: 9448 us
Latency quantile=0.5: 8728 us
Latency quantile=0.95: 15231 us
Latency quantile=0.99: 18009 us
Latency max: 24841 us
```
I also generated a 4096-bit RSA certificate. It's slightly faster, which surprises me.
```
Testing method: DoPut
Using standalone TCP server
Server host: ip-172-31-73-63.ec2.internal
Server port: 31337
Number of perf runs: 4
Number of concurrent gets/puts: 1
Batch size: 5265782
Batches written: 3456
Bytes written: 18198543232
Nanos: 40789196786
Speed: 425.492 MB/s
Throughput: 84.7283 batches/s
Latency mean: 9551 us
Latency quantile=0.5: 8684 us
Latency quantile=0.95: 15469 us
Latency quantile=0.99: 19123 us
Latency max: 36564 us
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #10242: ARROW-12648: [C++][FlightRPC] Enable TLS for Flight benchmark
Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #10242:
URL: https://github.com/apache/arrow/pull/10242#issuecomment-831985814
I tested this between two AWS EC2 instances (t3.xlarge) using a self-signed elliptic-curve certificate and one month's worth of the NYC taxi dataset. The performance impact is about 20%.
Plaintext:
```
$ arrow-flight-benchmark -server_host ip-172-31-73-63.ec2.internal -test_put -num_perf_runs=4 -num_streams=4 -num_threads=1 -data_file ~/data.feather
Testing method: DoPut
Using standalone TCP server
Server host: ip-172-31-73-63.ec2.internal
Server port: 31337
Number of perf runs: 4
Number of concurrent gets/puts: 1
Batch size: 5265782
Batches written: 3456
Bytes written: 18198543232
Nanos: 36934869371
Speed: 469.894 MB/s
Throughput: 93.5701 batches/s
Latency mean: 8501 us
Latency quantile=0.5: 8676 us
Latency quantile=0.95: 9189 us
Latency quantile=0.99: 10791 us
Latency max: 20421 us
```
TLS:
```
$ arrow-flight-benchmark -server_host ip-172-31-73-63.ec2.internal -test_put -num_perf_runs=4 -num_streams=4 -num_threads=1 -data_file ~/data.feather -cert_file cert.pem
Testing method: DoPut
Using standalone TCP server
Server host: ip-172-31-73-63.ec2.internal
Server port: 31337
Number of perf runs: 4
Number of concurrent gets/puts: 1
Batch size: 5265782
Batches written: 3456
Bytes written: 18198543232
Nanos: 43250718195
Speed: 401.276 MB/s
Throughput: 79.9062 batches/s
Latency mean: 9448 us
Latency quantile=0.5: 8728 us
Latency quantile=0.95: 15231 us
Latency quantile=0.99: 18009 us
Latency max: 24841 us
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org