You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/04/19 13:11:46 UTC
[GitHub] [flink-benchmarks] 1996fanrui opened a new pull request, #54: [FLINK-26762] Add the overdraft in checkpoint time benchmark
1996fanrui opened a new pull request, #54:
URL: https://github.com/apache/flink-benchmarks/pull/54
1. Add the overdraft in checkpoint time benchmark
2. Add the Param of sleepTime
3. Change MapFunction to FlatMapFunction
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-benchmarks] 1996fanrui commented on pull request #54: [FLINK-26762] Add the overdraft in checkpoint time benchmark
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on PR #54:
URL: https://github.com/apache/flink-benchmarks/pull/54#issuecomment-1103522433
Hi @akalash , the benchmark report is as follows:
Simple conclusion: UNALIGNED_OVERDRAFT performs better than UNALIGNED in FlatMap scene, especially when the backpressure is more severe (the sleepTime is greater), UNALIGNED_OVERDRAFT performs better.
```
Run on Linux Server, it's all Modes.
Benchmark (mode) (sleepTime) Mode Cnt Score Error Units
CheckpointingTimeBenchmark.checkpointSingleInput ALIGNED 10 ms thrpt 30 0.075 ± 0.002 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput ALIGNED 1 ms thrpt 30 0.622 ± 0.009 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput ALIGNED 200 µs thrpt 30 0.629 ± 0.011 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 10 ms thrpt 30 21.981 ± 3.934 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 1 ms thrpt 30 82.445 ± 10.198 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 200 µs thrpt 30 88.001 ± 13.431 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_1 10 ms thrpt 30 0.105 ± 0.005 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_1 1 ms thrpt 30 1.025 ± 0.097 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_1 200 µs thrpt 30 0.978 ± 0.067 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 10 ms thrpt 30 84.750 ± 7.647 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 1 ms thrpt 30 122.876 ± 13.307 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 200 µs thrpt 30 127.862 ± 10.773 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT_1 10 ms thrpt 30 0.107 ± 0.005 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT_1 1 ms thrpt 30 0.999 ± 0.072 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT_1 200 µs thrpt 30 0.970 ± 0.044 ops/s
Run on My Mac, I just run UNALIGNED and UNALIGNED_OVERDRAFT.
Benchmark (mode) (sleepTime) Mode Cnt Score Error Units
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 10 ms thrpt 30 25.077 ± 8.530 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 1 ms thrpt 30 262.581 ± 20.448 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 200 µs thrpt 30 220.085 ± 29.041 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 10 ms thrpt 30 80.799 ± 10.459 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 1 ms thrpt 30 580.895 ± 24.846 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 200 µs thrpt 30 584.682 ± 9.971 ops/s
```
### Some differences between Linux and Mac:
When sleepTime=10ms, the result of Linux and Mac are close, but I don't know why they have a big gap when sleepTime=1ms or 200µs? It may be caused by different CPUs.
### Common conclusion:
The result of UNALIGNED_OVERDRAFT is better than UNALIGNED no matter what sleepMs is.
- When sleepMs=1ms or 200µs, the performance of UNALIGNED_OVERDRAFT is 1.5 times that of UNALIGNED.
- When sleepMs=10ms, the performance of UNALIGNED_OVERDRAFT is more than 3 times that of UNALIGNED.
So the overdraft should be useful in scenarios where processing a single record requires multiple output buffers. Please correct my if any wrong, thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-benchmarks] 1996fanrui commented on pull request #54: [FLINK-26762] Add the overdraft in checkpoint time benchmark
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on PR #54:
URL: https://github.com/apache/flink-benchmarks/pull/54#issuecomment-1103807858
By the way, I found an interesting point while playing the benchmark. That is, the UNALIGNED_RECORD_SIZE is 1KB in the benchmark(In production, the single record size of many flink jobs is less than 1KB.). If UNALIGNED_RECORD_SIZE is reduced, UNALIGNED will perform worse, but UNALIGNED_OVERDRAFT performance is not affected.
Because flatmap requires 5 output buffers to process a single record, it will wait for 5 buffers to be processed downstream. If the record size is smaller, the 5 output buffers will hold more records, and the downstream Task will take longer to process these records. But UNALIGNED_OVERDRAFT allows buffers to be overdrawn, so the upstream does not need to wait for the downstream to process 5 buffers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-benchmarks] 1996fanrui commented on pull request #54: [FLINK-26762] Add the overdraft in checkpoint time benchmark
Posted by GitBox <gi...@apache.org>.
1996fanrui commented on PR #54:
URL: https://github.com/apache/flink-benchmarks/pull/54#issuecomment-1103521286
Hi @akalash , the benchmark report is as follows:
Simple conclusion: UNALIGNED_OVERDRAFT performs better than UNALIGNED in FlatMap scene, especially when the backpressure is more severe (the sleepTime is greater), UNALIGNED_OVERDRAFT performs better.
```
Run on Linux Server, it's all Modes.
Benchmark (mode) (sleepTime) Mode Cnt Score Error Units
CheckpointingTimeBenchmark.checkpointSingleInput ALIGNED 10 ms thrpt 30 0.075 ± 0.002 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput ALIGNED 1 ms thrpt 30 0.622 ± 0.009 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput ALIGNED 200 µs thrpt 30 0.629 ± 0.011 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 10 ms thrpt 30 21.981 ± 3.934 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 1 ms thrpt 30 82.445 ± 10.198 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 200 µs thrpt 30 88.001 ± 13.431 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_1 10 ms thrpt 30 0.105 ± 0.005 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_1 1 ms thrpt 30 1.025 ± 0.097 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_1 200 µs thrpt 30 0.978 ± 0.067 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 10 ms thrpt 30 84.750 ± 7.647 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 1 ms thrpt 30 122.876 ± 13.307 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 200 µs thrpt 30 127.862 ± 10.773 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT_1 10 ms thrpt 30 0.107 ± 0.005 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT_1 1 ms thrpt 30 0.999 ± 0.072 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT_1 200 µs thrpt 30 0.970 ± 0.044 ops/s
Run on My Mac, I just run UNALIGNED and UNALIGNED_OVERDRAFT.
Benchmark (mode) (sleepTime) Mode Cnt Score Error Units
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 10 ms thrpt 30 25.077 ± 8.530 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 1 ms thrpt 30 262.581 ± 20.448 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED 200 µs thrpt 30 220.085 ± 29.041 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 10 ms thrpt 30 80.799 ± 10.459 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 1 ms thrpt 30 580.895 ± 24.846 ops/s
CheckpointingTimeBenchmark.checkpointSingleInput UNALIGNED_OVERDRAFT 200 µs thrpt 30 584.682 ± 9.971 ops/s
```
### Some differences between Linux and Mac:
When sleepTime=10ms, the result of Linux and Mac are close, but I don't know why they have a big gap when sleepTime=1ms or 200µs? It may be caused by different CPUs.
### Common conclusion:
The result of UNALIGNED_OVERDRAFT is better than UNALIGNED no matter what sleepMs is.
- When sleepMs=1ms or 200µs, the performance of UNALIGNED_OVERDRAFT is 1.5 times that of UNALIGNED.
- When sleepMs=10ms, the performance of UNALIGNED_OVERDRAFT is more than 3 times that of UNALIGNED.
So It might be caused by different CPUs.
Common conclusion:
The result of UNALIGNED_OVERDRAFT is better than UNALIGNED no matter what sleepMs is.
When sleepMs=1ms or 200ms, the performance of UNALIGNED_OVERDRAFT is 1.5 times that of UNALIGNED.
When sleepMs=10ms, the performance of UNALIGNED_OVERDRAFT is more than 3 times that of UNALIGNED.
So the overdraft should be useful in scenarios where processing a single record requires multiple output buffers. Please correct my if any wrong, thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-benchmarks] 1996fanrui closed pull request #54: [FLINK-26762] Add the overdraft in checkpoint time benchmark
Posted by GitBox <gi...@apache.org>.
1996fanrui closed pull request #54: [FLINK-26762] Add the overdraft in checkpoint time benchmark
URL: https://github.com/apache/flink-benchmarks/pull/54
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org