You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Piotr Nowojski (JIRA)" <ji...@apache.org> on 2019/08/12 10:21:00 UTC
[jira] [Commented] (FLINK-12818) Improve stability of
twoInputMapSink benchmark
[ https://issues.apache.org/jira/browse/FLINK-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905051#comment-16905051 ]
Piotr Nowojski commented on FLINK-12818:
----------------------------------------
Thanks for the investigation. I tried running with your configuration and with disabled checkpointing, but my local results were even less stable:
{noformat}
# Run progress: 0.00% complete, ETA 00:00:24
# Fork: 1 of 3
# Warmup Iteration 1: 5428.001 ops/ms
# Warmup Iteration 2: 9326.627 ops/ms
# Warmup Iteration 3: 10375.921 ops/ms
Iteration 1: 11318.975 ops/ms
Iteration 2: 10919.210 ops/ms
Iteration 3: 11385.420 ops/ms
Iteration 4: 11588.595 ops/ms
Iteration 5: 11699.230 ops/ms
# Run progress: 33.33% complete, ETA 00:00:44
# Fork: 2 of 3
# Warmup Iteration 1: 6764.317 ops/ms
# Warmup Iteration 2: 12866.327 ops/ms
# Warmup Iteration 3: 13283.575 ops/ms
Iteration 1: 18927.883 ops/ms
Iteration 2: 18005.842 ops/ms
Iteration 3: 16681.179 ops/ms
Iteration 4: 20368.397 ops/ms
Iteration 5: 16163.679 ops/ms
# Run progress: 66.67% complete, ETA 00:00:18
# Fork: 3 of 3
# Warmup Iteration 1: 6622.793 ops/ms
# Warmup Iteration 2: 12608.261 ops/ms
# Warmup Iteration 3: 14369.082 ops/ms
Iteration 1: 14280.980 ops/ms
Iteration 2: 14649.605 ops/ms
Iteration 3: 14222.255 ops/ms
Iteration 4: 14735.759 ops/ms
Iteration 5: 14227.849 ops/ms
{noformat}
Also disabling the checkpointing also removes benchmark coverage in a code that has changed significantly between selectable and non-selectable TwoInputOperators.
I think if we want to investigate this further, we would need to print compiled byte code and try to understand the difference.
Alternatively until this is confirmed as a production code issue, we can just try to live with this benchmark instability and relay on long term trends.
> Improve stability of twoInputMapSink benchmark
> ----------------------------------------------
>
> Key: FLINK-12818
> URL: https://issues.apache.org/jira/browse/FLINK-12818
> Project: Flink
> Issue Type: Sub-task
> Components: Benchmarks
> Reporter: Piotr Nowojski
> Priority: Critical
> Attachments: RecordWriter-emit.png
>
>
> The {{twoInputMapSink}} benchmark is very unstable over time:
> http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=twoInputMapSink&env=2&revs=200&equid=off&quarts=on&extr=on
> It should be fixed, otherwise the benchmark can not be used.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)