You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2022/07/12 11:55:56 UTC
Beam Metrics Report (2022-07-12)
# Beam Metrics Report
Color legend: >= 20 | >= 10 | <= -10 | <= -20
---|---|---|---
## Possible regression
**Measurement** | **Metric** | **Runner** | **Mean previous week** | **Mean
last week** | **Diff %** | **Dashboard**
---|---|---|---|---|---|---
java_batch_cogbk_3 | dataflow_v2_java17_runtime_sec | - | 13.58 | 16.39 | 20.7
| [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_4 | dataflow_v2_java11_runtime_sec | - | 14.25 | 15.78 |
10.76 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_gbk_3 | dataflow_v2_java11_runtime_sec | - | 14.44 | 17.31 | 19.88
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_7 | dataflow_v2_java17_runtime_sec | - | 35.86 | 39.61 | 10.45
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_7 | sparkstructuredstreaming_runtime_sec | - | 84.12 | 92.67 |
10.17 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_pardo_1 | dataflow_v2_java11_runtime_sec | - | 13.25 | 16.08 |
21.34 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_1 | dataflow_v2_java17_runtime_sec | - | 15.09 | 16.7 | 10.66
| [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_2 | dataflow_v2_java11_runtime_sec | - | 63.83 | 71.69 |
12.31 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_3 | dataflow_v2_java11_runtime_sec | - | 9.09 | 15.75 | 73.32
| [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_3 | dataflow_v2_java17_runtime_sec | - | 12.12 | 14.5 | 19.67
| [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_gbk_3 | dataflow_v2_java11_runtime_sec | - | 53.16 | 59.27 |
11.48 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_pardo_3 | dataflow_v2_java11_runtime_sec | - | 21.3 | 23.56 |
10.62 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
kafkaioit_results_runner_v2 | read_time | - | 1.87 | 2.36 | 26.33 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
python_batch_cogbk_2 | python_dataflow_batch_cogbk_2_runtime | - | 287.43 |
317.2 | 10.36 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_batch_gbk_3 | python_dataflow_batch_gbk_3_runtime | - | 27.43 | 32.6 |
18.85 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_streaming_pardo_4 | python_dataflow_streaming_pardo_4_runtime | - |
538.57 | 602.0 | 11.78 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
nexmark_11_batch | RuntimeMs | DataflowRunner | 39589.51 | 61325.57 | 54.9 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_16_batch | RuntimeMs | FlinkRunner | 214.46 | 256.23 | 19.47 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_7_sql_batch | RuntimeMs | DirectRunner | 326788.15 | 364568.64 | 11.56
| [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_7_sql_streaming | RuntimeMs | DirectRunner | 1076402.96 | 1215625.41 |
12.93 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_9_batch | RuntimeMs | FlinkRunner | 290.18 | 325.45 | 12.16 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_9_streaming | RuntimeMs | FlinkRunner | 430.46 | 487.23 | 13.19 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
## Possible improvement
**Measurement** | **Metric** | **Runner** | **Mean previous week** | **Mean
last week** | **Diff %** | **Dashboard**
---|---|---|---|---|---|---
go_batch_gbk_7 | dataflow_runtime | - | 125.77 | 111.91 | -11.02 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
go_batch_pardo_2 | dataflow_runtime | - | 420.66 | 360.73 | -14.25 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_cogbk_2 | dataflow_v2_java11_runtime_sec | - | 50.65 | 44.37 |
-12.41 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_gbk_3 | dataflow_runtime_sec | - | 11.22 | 10.07 | -10.3 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_6 | dataflow_v2_java17_runtime_sec | - | 31.33 | 26.3 | -16.06
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_7 | dataflow_runtime_sec | - | 49.25 | 44.03 | -10.62 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_7 | dataflow_v2_java11_runtime_sec | - | 42.23 | 34.58 | -18.11
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_pardo_1 | sparkstructuredstreaming_runtime_sec | - | 56.17 | 49.55
| -11.8 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_3 | dataflow_runtime_sec | - | 13.8 | 12.29 | -10.98 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_cogbk_1 | dataflow_v2_java11_runtime_sec | - | 3095.03 |
2628.35 | -15.08 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_gbk_1 | dataflow_v2_java17_runtime_sec | - | 1003.04 | 897.89 |
-10.48 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_2 | dataflow_v2_java11_runtime_sec | - | 4656.02 | 2798.25
| -39.9 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_3 | dataflow_v2_java17_runtime_sec | - | 69.2 | 53.75 |
-22.32 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_5 | dataflow_v2_java17_runtime_sec | - | 677.63 | 555.04 |
-18.09 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_pardo_1 | dataflow_v2_java11_runtime_sec | - | 28.86 | 23.28 |
-19.35 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_4 | dataflow_v2_java17_runtime_sec | - | 37.98 | 33.86 |
-10.85 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
kafkaioit_results | read_time | - | 777.55 | 0.0 | -100.0 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
kafkaioit_results | write_time | - | 212.65 | 14.37 | -93.24 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
kafkaioit_results_sdf_wrapper | write_time | - | 6.25 | 3.97 | -36.43 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
python_batch_cogbk_1 | python_dataflow_batch_cogbk_1_runtime | - | 525.29 |
453.8 | -13.61 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_batch_cogbk_3 | python_dataflow_batch_cogbk_3_runtime | - | 282.71 |
253.8 | -10.23 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_batch_combine_4 | python_dataflow_batch_combine_4_runtime | - | 61.86 |
53.0 | -14.32 | [[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
python_batch_sideinput_2 | python_dataflow_batch_sideinput_2_runtime | - |
50.14 | 45.0 | -10.26 | [[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
python_batch_sideinput_6 | python_dataflow_batch_sideinput_6_runtime | - |
45.29 | 38.0 | -16.09 | [[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
python_bqio_write_10GB_results | bqio_write_10GB_results_runtime | - | 247.86
| 210.2 | -15.19 | [[1]](http://metrics.beam.apache.org/d/gP7vMPqZz)
python_streaming_gbk_3 | python_dataflow_streaming_gbk_3_runtime | - | 90.86 |
78.5 | -13.6 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
nexmark_11_batch | RuntimeMs | SparkStructuredStreamingRunner | 1135.95 |
1015.14 | -10.64 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_13_sql_streaming | RuntimeMs | DataflowRunner | 42997.25 | 34577.23 |
-19.58 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_13_streaming | RuntimeMs | DataflowRunner | 28812.61 | 24063.0 |
-16.48 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_15_batch | RuntimeMs | FlinkRunner | 646.39 | 579.05 | -10.42 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_2_batch | RuntimeMs | FlinkRunner | 219.18 | 192.68 | -12.09 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
## All metrics
**Measurement** | **Metric** | **Runner** | **Mean previous week** | **Mean
last week** | **Diff %** | **Dashboard**
---|---|---|---|---|---|---
avroioit_hdfs_results | read_time | - | 88.5 | 90.3 | 2.04 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
avroioit_hdfs_results | write_time | - | 117.12 | 121.15 | 3.43 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
avroioit_results | read_time | - | 90.57 | 93.45 | 3.18 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
avroioit_results | write_time | - | 116.22 | 119.98 | 3.23 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
bqio_10GB_results_java_batch_avro | avro_write_time | - | 35.46 | 34.18 |
-3.61 | [[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
bqio_10GB_results_java_batch_avro | read_time | - | 41.22 | 41.64 | 1.02 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
bqio_10GB_results_java_batch_json | read_time | - | 42.36 | 38.94 | -8.07 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
bqio_10GB_results_java_batch_json | write_time | - | 51.59 | 50.42 | -2.26 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
compressed_textioit_hdfs_results | read_time | - | 158.32 | 160.91 | 1.64 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
compressed_textioit_hdfs_results | write_time | - | 207.37 | 207.08 | -0.14 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
compressed_textioit_results | read_time | - | 158.41 | 161.48 | 1.94 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
compressed_textioit_results | write_time | - | 204.95 | 204.34 | -0.3 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
go_batch_cogbk_1 | dataflow_runtime | - | 256.19 | 251.49 | -1.84 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
go_batch_cogbk_2 | dataflow_runtime | - | 148.89 | 143.6 | -3.55 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
go_batch_cogbk_3 | dataflow_runtime | - | 115.23 | 112.94 | -1.99 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
go_batch_cogbk_4 | dataflow_runtime | - | 114.24 | 112.66 | -1.38 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
go_batch_combine_1 | dataflow_runtime | - | 388.21 | 377.26 | -2.82 |
[[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
go_batch_combine_4 | dataflow_runtime | - | 77.88 | 76.26 | -2.09 |
[[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
go_batch_combine_5 | dataflow_runtime | - | 72.59 | 71.3 | -1.78 |
[[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
go_batch_gbk_1 | dataflow_runtime | - | 371.28 | 369.86 | -0.38 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
go_batch_gbk_2 | dataflow_runtime | - | 100.97 | 98.34 | -2.6 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
go_batch_gbk_3 | dataflow_runtime | - | 31.82 | 30.17 | -5.18 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
go_batch_gbk_4 | dataflow_runtime | - | 55.12 | 54.68 | -0.78 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
go_batch_gbk_5 | dataflow_runtime | - | 45.18 | 45.82 | 1.42 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
go_batch_gbk_6 | dataflow_runtime | - | 102.59 | 103.03 | 0.44 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
go_batch_gbk_7 | dataflow_runtime | - | 125.77 | 111.91 | -11.02 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
go_batch_pardo_1 | dataflow_runtime | - | 75.89 | 78.22 | 3.07 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
go_batch_pardo_2 | dataflow_runtime | - | 420.66 | 360.73 | -14.25 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
go_batch_pardo_3 | dataflow_runtime | - | 63.86 | 65.18 | 2.07 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
go_batch_pardo_4 | dataflow_runtime | - | 72.36 | 70.43 | -2.67 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
go_batch_sideinput_3 | dataflow_runtime | - | 2.35 | 2.44 | 3.86 |
[[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
go_batch_sideinput_4 | dataflow_runtime | - | 208.13 | 206.1 | -0.98 |
[[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
hadoopformatioit_results | read_time | - | 6.33 | 6.34 | 0.13 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
hadoopformatioit_results | write_time | - | 7.79 | 7.51 | -3.63 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
java_batch_cogbk_1 | dataflow_runtime_sec | - | 41.75 | 42.28 | 1.27 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_1 | dataflow_v2_java11_runtime_sec | - | 46.39 | 43.44 |
-6.36 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_1 | dataflow_v2_java17_runtime_sec | - | 49.57 | 45.83 |
-7.54 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_1 | sparkstructuredstreaming_runtime_sec | - | 118.05 |
124.41 | 5.39 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_2 | dataflow_runtime_sec | - | 39.63 | 42.27 | 6.67 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_2 | dataflow_v2_java11_runtime_sec | - | 50.65 | 44.37 |
-12.41 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_2 | dataflow_v2_java17_runtime_sec | - | 44.58 | 44.66 | 0.18
| [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_2 | sparkstructuredstreaming_runtime_sec | - | 116.79 |
122.12 | 4.56 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_3 | dataflow_runtime_sec | - | 15.17 | 15.82 | 4.32 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_3 | dataflow_v2_java11_runtime_sec | - | 14.41 | 13.74 |
-4.68 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_3 | dataflow_v2_java17_runtime_sec | - | 13.58 | 16.39 | 20.7
| [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_3 | sparkstructuredstreaming_runtime_sec | - | 19.96 | 20.61
| 3.28 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_4 | dataflow_runtime_sec | - | 13.12 | 13.65 | 4.05 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_4 | dataflow_v2_java11_runtime_sec | - | 14.25 | 15.78 |
10.76 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_4 | dataflow_v2_java17_runtime_sec | - | 13.23 | 13.71 | 3.59
| [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_cogbk_4 | sparkstructuredstreaming_runtime_sec | - | 20.37 | 20.12
| -1.19 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_batch_combine_1 | dataflow_runtime_sec | - | 64.0 | 62.62 | -2.16 |
[[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
java_batch_combine_1 | sparkstructuredstreaming_runtime_sec | - | 554.59 |
557.39 | 0.5 | [[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
java_batch_combine_4 | dataflow_runtime_sec | - | 27.99 | 27.76 | -0.8 |
[[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
java_batch_combine_4 | sparkstructuredstreaming_runtime_sec | - | 124.67 |
126.83 | 1.73 | [[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
java_batch_combine_5 | dataflow_runtime_sec | - | 27.27 | 24.84 | -8.93 |
[[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
java_batch_combine_5 | sparkstructuredstreaming_runtime_sec | - | 119.92 |
120.06 | 0.12 | [[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
java_batch_gbk_1 | dataflow_runtime_sec | - | 125.42 | 115.83 | -7.64 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_1 | dataflow_v2_java11_runtime_sec | - | 96.68 | 95.35 | -1.38
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_1 | dataflow_v2_java17_runtime_sec | - | 86.51 | 90.35 | 4.44 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_1 | sparkstructuredstreaming_runtime_sec | - | 502.91 | 503.81
| 0.18 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_2 | dataflow_runtime_sec | - | 33.68 | 32.33 | -4.0 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_2 | dataflow_v2_java11_runtime_sec | - | 42.27 | 40.93 | -3.18
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_2 | dataflow_v2_java17_runtime_sec | - | 39.31 | 40.94 | 4.12 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_2 | sparkstructuredstreaming_runtime_sec | - | 97.8 | 107.11 |
9.52 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_3 | dataflow_runtime_sec | - | 11.22 | 10.07 | -10.3 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_3 | dataflow_v2_java11_runtime_sec | - | 14.44 | 17.31 | 19.88
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_3 | dataflow_v2_java17_runtime_sec | - | 15.28 | 15.39 | 0.75 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_3 | sparkstructuredstreaming_runtime_sec | - | 8.2 | 8.74 |
6.56 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_4 | dataflow_runtime_sec | - | 18.85 | 19.23 | 2.05 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_4 | dataflow_v2_java11_runtime_sec | - | 22.58 | 22.18 | -1.78
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_4 | dataflow_v2_java17_runtime_sec | - | 23.02 | 24.62 | 6.92 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_4 | sparkstructuredstreaming_runtime_sec | - | 97.45 | 101.76 |
4.43 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_5 | dataflow_runtime_sec | - | 18.5 | 17.83 | -3.64 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_5 | dataflow_v2_java11_runtime_sec | - | 17.53 | 17.45 | -0.49
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_5 | dataflow_v2_java17_runtime_sec | - | 19.43 | 20.54 | 5.68 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_5 | sparkstructuredstreaming_runtime_sec | - | 100.52 | 100.52
| -0.0 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_6 | dataflow_runtime_sec | - | 37.2 | 34.88 | -6.26 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_6 | dataflow_v2_java11_runtime_sec | - | 26.4 | 26.36 | -0.12 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_6 | dataflow_v2_java17_runtime_sec | - | 31.33 | 26.3 | -16.06
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_6 | sparkstructuredstreaming_runtime_sec | - | 58.34 | 60.86 |
4.32 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_7 | dataflow_runtime_sec | - | 49.25 | 44.03 | -10.62 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_7 | dataflow_v2_java11_runtime_sec | - | 42.23 | 34.58 | -18.11
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_7 | dataflow_v2_java17_runtime_sec | - | 35.86 | 39.61 | 10.45
| [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_gbk_7 | sparkstructuredstreaming_runtime_sec | - | 84.12 | 92.67 |
10.17 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_batch_pardo_1 | dataflow_runtime_sec | - | 20.04 | 18.14 | -9.49 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_1 | dataflow_v2_java11_runtime_sec | - | 13.25 | 16.08 |
21.34 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_1 | dataflow_v2_java17_runtime_sec | - | 15.09 | 16.7 | 10.66
| [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_1 | sparkstructuredstreaming_runtime_sec | - | 56.17 | 49.55
| -11.8 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_2 | dataflow_runtime_sec | - | 186.08 | 202.59 | 8.87 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_2 | dataflow_v2_java11_runtime_sec | - | 63.83 | 71.69 |
12.31 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_2 | dataflow_v2_java17_runtime_sec | - | 66.38 | 67.13 | 1.13
| [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_3 | dataflow_runtime_sec | - | 13.8 | 12.29 | -10.98 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_3 | dataflow_v2_java11_runtime_sec | - | 9.09 | 15.75 | 73.32
| [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_3 | dataflow_v2_java17_runtime_sec | - | 12.12 | 14.5 | 19.67
| [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_4 | dataflow_runtime_sec | - | 24.8 | 22.52 | -9.18 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_4 | dataflow_v2_java11_runtime_sec | - | 26.58 | 24.46 |
-7.98 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_batch_pardo_4 | dataflow_v2_java17_runtime_sec | - | 25.41 | 24.93 | -1.9
| [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_cogbk_1 | dataflow_runtime_sec | - | 927.84 | 858.74 | -7.45 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_1 | dataflow_v2_java11_runtime_sec | - | 3095.03 |
2628.35 | -15.08 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_1 | dataflow_v2_java17_runtime_sec | - | 2589.29 |
2595.64 | 0.25 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_2 | dataflow_runtime_sec | - | 907.41 | 837.58 | -7.7 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_2 | dataflow_v2_java11_runtime_sec | - | 3256.56 |
2969.53 | -8.81 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_2 | dataflow_v2_java17_runtime_sec | - | 2938.01 |
3090.66 | 5.2 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_3 | dataflow_runtime_sec | - | 118.04 | 122.42 | 3.71 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_3 | dataflow_v2_java11_runtime_sec | - | 322.79 | 326.12
| 1.03 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_3 | dataflow_v2_java17_runtime_sec | - | 319.04 | 325.44
| 2.01 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_4 | dataflow_runtime_sec | - | 125.95 | 122.06 | -3.09 |
[[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_4 | dataflow_v2_java11_runtime_sec | - | 320.92 | 326.47
| 1.73 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_cogbk_4 | dataflow_v2_java17_runtime_sec | - | 331.9 | 322.66 |
-2.78 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
java_streaming_combine_1 | dataflow_runtime_sec | - | 509.03 | 511.79 | 0.54 |
[[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
java_streaming_combine_4 | dataflow_runtime_sec | - | 318.94 | 322.22 | 1.03 |
[[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
java_streaming_combine_5 | dataflow_runtime_sec | - | 307.67 | 309.5 | 0.6 |
[[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
java_streaming_gbk_1 | dataflow_runtime_sec | - | 670.68 | 646.17 | -3.65 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_1 | dataflow_v2_java11_runtime_sec | - | 875.0 | 920.0 |
5.14 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_1 | dataflow_v2_java17_runtime_sec | - | 1003.04 | 897.89 |
-10.48 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_2 | dataflow_runtime_sec | - | 803.33 | 798.42 | -0.61 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_2 | dataflow_v2_java11_runtime_sec | - | 4656.02 | 2798.25
| -39.9 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_2 | dataflow_v2_java17_runtime_sec | - | 3158.86 | 3166.16
| 0.23 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_3 | dataflow_runtime_sec | - | 23.62 | 24.92 | 5.49 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_3 | dataflow_v2_java11_runtime_sec | - | 53.16 | 59.27 |
11.48 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_3 | dataflow_v2_java17_runtime_sec | - | 69.2 | 53.75 |
-22.32 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_4 | dataflow_runtime_sec | - | 289.5 | 286.49 | -1.04 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_4 | dataflow_v2_java11_runtime_sec | - | 641.36 | 649.91 |
1.33 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_4 | dataflow_v2_java17_runtime_sec | - | 665.68 | 685.61 |
2.99 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_5 | dataflow_runtime_sec | - | 293.31 | 280.95 | -4.21 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_5 | dataflow_v2_java11_runtime_sec | - | 602.18 | 632.83 |
5.09 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_5 | dataflow_v2_java17_runtime_sec | - | 677.63 | 555.04 |
-18.09 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_6 | dataflow_runtime_sec | - | 73.86 | 76.09 | 3.02 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_6 | dataflow_v2_java11_runtime_sec | - | 160.18 | 145.57 |
-9.12 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_6 | dataflow_v2_java17_runtime_sec | - | 194.24 | 178.72 |
-7.99 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_7 | dataflow_runtime_sec | - | 137.68 | 137.62 | -0.04 |
[[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_7 | dataflow_v2_java11_runtime_sec | - | 207.62 | 226.19 |
8.94 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_gbk_7 | dataflow_v2_java17_runtime_sec | - | 223.14 | 202.22 |
-9.37 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
java_streaming_pardo_1 | dataflow_runtime_sec | - | 18.79 | 18.93 | 0.75 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_1 | dataflow_v2_java11_runtime_sec | - | 28.86 | 23.28 |
-19.35 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_1 | dataflow_v2_java17_runtime_sec | - | 25.27 | 24.23 |
-4.08 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_2 | dataflow_runtime_sec | - | 223.51 | 226.23 | 1.22 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_2 | dataflow_v2_java11_runtime_sec | - | 114.31 | 111.55
| -2.41 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_2 | dataflow_v2_java17_runtime_sec | - | 105.51 | 97.51 |
-7.58 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_3 | dataflow_runtime_sec | - | 12.55 | 12.71 | 1.25 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_3 | dataflow_v2_java11_runtime_sec | - | 21.3 | 23.56 |
10.62 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_3 | dataflow_v2_java17_runtime_sec | - | 24.03 | 24.17 |
0.61 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_4 | dataflow_runtime_sec | - | 32.98 | 33.47 | 1.49 |
[[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_4 | dataflow_v2_java11_runtime_sec | - | 36.78 | 35.46 |
-3.61 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
java_streaming_pardo_4 | dataflow_v2_java17_runtime_sec | - | 37.98 | 33.86 |
-10.85 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
jdbcioit_results | read_time | - | 24.89 | 24.82 | -0.28 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
jdbcioit_results | write_time | - | 38.16 | 36.52 | -4.3 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
kafkaioit_results | read_time | - | 777.55 | 0.0 | -100.0 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
kafkaioit_results | write_time | - | 212.65 | 14.37 | -93.24 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
kafkaioit_results_runner_v2 | read_time | - | 1.87 | 2.36 | 26.33 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
kafkaioit_results_runner_v2 | write_time | - | 5.89 | 5.91 | 0.4 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
kafkaioit_results_sdf_wrapper | read_time | - | 3.04 | 3.26 | 7.08 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
kafkaioit_results_sdf_wrapper | write_time | - | 6.25 | 3.97 | -36.43 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
many_files_textioit_hdfs_results | read_time | - | 17.29 | 17.48 | 1.13 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
many_files_textioit_hdfs_results | write_time | - | 61.16 | 56.82 | -7.09 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
many_files_textioit_results | read_time | - | 24.9 | 24.23 | -2.67 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
many_files_textioit_results | write_time | - | 61.41 | 58.16 | -5.29 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
mongodbioit_results | read_time | - | 30.18 | 30.83 | 2.15 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
mongodbioit_results | write_time | - | 59.02 | 55.45 | -6.06 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
parquetioit_hdfs_results | read_time | - | 88.2 | 88.97 | 0.88 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
parquetioit_hdfs_results | write_time | - | 147.1 | 142.12 | -3.39 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
parquetioit_results | read_time | - | 87.88 | 89.9 | 2.29 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
parquetioit_results | write_time | - | 145.06 | 139.23 | -4.02 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
python_batch_cogbk_1 | python_dataflow_batch_cogbk_1_runtime | - | 525.29 |
453.8 | -13.61 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_batch_cogbk_2 | python_dataflow_batch_cogbk_2_runtime | - | 287.43 |
317.2 | 10.36 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_batch_cogbk_3 | python_dataflow_batch_cogbk_3_runtime | - | 282.71 |
253.8 | -10.23 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_batch_cogbk_4 | python_dataflow_batch_cogbk_4_runtime | - | 291.71 |
278.4 | -4.56 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_batch_combine_1 | python_dataflow_batch_combine_1_runtime | - | 841.29
| 875.6 | 4.08 | [[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
python_batch_combine_4 | python_dataflow_batch_combine_4_runtime | - | 61.86 |
53.0 | -14.32 | [[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
python_batch_combine_5 | python_dataflow_batch_combine_5_runtime | - | 46.0 |
44.4 | -3.48 | [[1]](http://metrics.beam.apache.org/d/WNzYt13Zk)
python_batch_gbk_1 | python_dataflow_batch_gbk_1_runtime | - | 1877.29 |
1915.8 | 2.05 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_batch_gbk_2 | python_dataflow_batch_gbk_2_runtime | - | 373.14 | 367.8
| -1.43 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_batch_gbk_3 | python_dataflow_batch_gbk_3_runtime | - | 27.43 | 32.6 |
18.85 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_batch_gbk_4 | python_dataflow_batch_gbk_4_runtime | - | 124.43 | 124.2
| -0.18 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_batch_gbk_5 | python_dataflow_batch_gbk_5_runtime | - | 127.0 | 129.4 |
1.89 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_batch_gbk_6 | python_dataflow_batch_gbk_6_runtime | - | 344.86 | 352.4
| 2.19 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_batch_gbk_7 | python_dataflow_batch_gbk_7_runtime | - | 353.14 | 346.0
| -2.02 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_batch_pardo_1 | python_dataflow_batch_pardo_1_runtime | - | 180.14 |
176.8 | -1.86 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
python_batch_pardo_2 | python_dataflow_batch_pardo_2_runtime | - | 1635.57 |
1661.4 | 1.58 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
python_batch_pardo_3 | python_dataflow_batch_pardo_3_runtime | - | 141.86 |
150.6 | 6.16 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
python_batch_pardo_4 | python_dataflow_batch_pardo_4_runtime | - | 326.71 |
345.2 | 5.66 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
python_batch_sideinput_1 | python_dataflow_batch_sideinput_1_runtime | - |
47.86 | 46.4 | -3.04 | [[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
python_batch_sideinput_2 | python_dataflow_batch_sideinput_2_runtime | - |
50.14 | 45.0 | -10.26 | [[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
python_batch_sideinput_3 | python_dataflow_batch_sideinput_3_runtime | - |
5.71 | 5.4 | -5.5 | [[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
python_batch_sideinput_4 | python_dataflow_batch_sideinput_4_runtime | - |
471.29 | 436.4 | -7.4 | [[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
python_batch_sideinput_5 | python_dataflow_batch_sideinput_5_runtime | - |
47.29 | 44.4 | -6.1 | [[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
python_batch_sideinput_6 | python_dataflow_batch_sideinput_6_runtime | - |
45.29 | 38.0 | -16.09 | [[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
python_batch_sideinput_7 | python_dataflow_batch_sideinput_7_runtime | - |
5566.29 | 5588.2 | 0.39 | [[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
python_batch_sideinput_8 | python_dataflow_batch_sideinput_8_runtime | - |
5592.14 | 5562.8 | -0.52 | [[1]](http://metrics.beam.apache.org/d/-E9aGlFGk)
python_bqio_read_10GB_results | bqio_read_10GB_results_runtime | - | 118.71 |
111.6 | -5.99 | [[1]](http://metrics.beam.apache.org/d/gP7vMPqZz)
python_bqio_write_10GB_results | bqio_write_10GB_results_runtime | - | 247.86
| 210.2 | -15.19 | [[1]](http://metrics.beam.apache.org/d/gP7vMPqZz)
python_direct_microbenchmarks | fn_api_runner_microbenchmark_runtime_sec | - |
94.83 | 94.11 | -0.76 |
python_direct_microbenchmarks | teststream_microbenchmark_runtime_sec | - |
344.58 | 337.86 | -1.95 |
python_spanner_write_2GB_results | pyspanner_write_2GB_results_runtime | - |
1562.57 | 1599.2 | 2.34 | [[1]](http://metrics.beam.apache.org/d/gP7vMPqZz)
python_spannerio_read_2GB_results | pyspanner_read_2GB_results_runtime | - |
26.43 | 23.8 | -9.95 | [[1]](http://metrics.beam.apache.org/d/gP7vMPqZz)
python_streaming_cogbk_1 | python_dataflow_streaming_cogbk_1_runtime | - |
628.29 | 659.8 | 5.02 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_streaming_cogbk_2 | python_dataflow_streaming_cogbk_2_runtime | - |
425.0 | 422.4 | -0.61 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_streaming_cogbk_3 | python_dataflow_streaming_cogbk_3_runtime | - |
506.43 | 491.2 | -3.01 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_streaming_cogbk_4 | python_dataflow_streaming_cogbk_4_runtime | - |
424.71 | 427.0 | 0.54 | [[1]](http://metrics.beam.apache.org/d/fK0U4JqWz)
python_streaming_gbk_3 | python_dataflow_streaming_gbk_3_runtime | - | 90.86 |
78.5 | -13.6 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_streaming_gbk_6 | python_dataflow_streaming_gbk_6_runtime | - | 523.57
| 546.0 | 4.28 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_streaming_gbk_7 | python_dataflow_streaming_gbk_7_runtime | - | 506.29
| 553.5 | 9.33 | [[1]](http://metrics.beam.apache.org/d/UYZ-oJ3Zk)
python_streaming_pardo_1 | python_dataflow_streaming_pardo_1_runtime | - |
253.57 | 235.5 | -7.13 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
python_streaming_pardo_2 | python_dataflow_streaming_pardo_2_runtime | - |
2941.57 | 3096.5 | 5.27 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
python_streaming_pardo_3 | python_dataflow_streaming_pardo_3_runtime | - |
204.0 | 200.0 | -1.96 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
python_streaming_pardo_4 | python_dataflow_streaming_pardo_4_runtime | - |
538.57 | 602.0 | 11.78 | [[1]](http://metrics.beam.apache.org/d/MOi-kf3Zk)
textioit_hdfs_results | read_time | - | 19.88 | 20.73 | 4.27 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
textioit_hdfs_results | write_time | - | 32.34 | 31.18 | -3.59 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
textioit_results | read_time | - | 20.52 | 21.58 | 5.16 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
textioit_results | write_time | - | 32.12 | 32.81 | 2.14 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
tfrecordioit_results | read_time | - | 15.43 | 16.26 | 5.35 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
tfrecordioit_results | write_time | - | 20.79 | 21.14 | 1.68 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
wordcount_py37_results | runtime | - | 811.08 | 824.06 | 1.6 |
[[1]](http://metrics.beam.apache.org/d/1cnwVDkGk)
xmlioit_hdfs_results | read_time | - | 21.4 | 22.77 | 6.44 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
xmlioit_hdfs_results | write_time | - | 24.67 | 25.12 | 1.79 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
xmlioit_results | read_time | - | 23.48 | 23.36 | -0.53 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
xmlioit_results | write_time | - | 24.44 | 26.24 | 7.36 |
[[1]](http://metrics.beam.apache.org/d/bnlHKP3Wz)
nexmark_0_batch | RuntimeMs | DataflowRunner | 27146.02 | 27212.08 | 0.24 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_0_batch | RuntimeMs | DirectRunner | 4782.36 | 4783.0 | 0.01 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_0_batch | RuntimeMs | FlinkRunner | 1155.07 | 1111.86 | -3.74 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_0_batch | RuntimeMs | SparkRunner | 1253.64 | 1255.18 | 0.12 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_0_batch | RuntimeMs | SparkStructuredStreamingRunner | 1546.91 |
1554.18 | 0.47 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_0_sql_batch | RuntimeMs | DataflowRunner | 35274.21 | 35610.5 | 0.95 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_0_sql_batch | RuntimeMs | DirectRunner | 5570.26 | 5655.59 | 1.53 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_0_sql_batch | RuntimeMs | SparkRunner | 3205.2 | 3141.48 | -1.99 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_0_sql_batch | RuntimeMs | SparkStructuredStreamingRunner | 2939.46 |
2926.11 | -0.45 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_0_sql_streaming | RuntimeMs | DataflowRunner | 24026.89 | 23121.55 |
-3.77 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_0_sql_streaming | RuntimeMs | DirectRunner | 15501.0 | 15555.91 | 0.35
| [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_0_sql_streaming | RuntimeMs | FlinkRunner | 1637.46 | 1628.64 | -0.54
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_0_streaming | RuntimeMs | DataflowRunner | 39291.92 | 37816.14 | -3.76
| [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_0_streaming | RuntimeMs | DirectRunner | 11811.79 | 11938.45 | 1.07 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_0_streaming | RuntimeMs | FlinkRunner | 1432.14 | 1422.36 | -0.68 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_0_zetasql_batch | RuntimeMs | DataflowRunner | 114915.96 | 112238.45 |
-2.33 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_0_zetasql_batch | RuntimeMs | DirectRunner | 6757.07 | 6834.27 | 1.14
| [[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_0_zetasql_streaming | RuntimeMs | DataflowRunner | 67751.04 | 66627.91
| -1.66 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_0_zetasql_streaming | RuntimeMs | DirectRunner | 17105.75 | 16996.45 |
-0.64 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_0_zetasql_streaming | RuntimeMs | FlinkRunner | 3248.54 | 3261.41 |
0.4 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_10_batch | RuntimeMs | DataflowRunner | 48690.3 | 49306.87 | 1.27 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_10_batch | RuntimeMs | DirectRunner | 10463.32 | 10465.09 | 0.02 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_10_batch | RuntimeMs | FlinkRunner | 777.21 | 714.64 | -8.05 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_10_batch | RuntimeMs | SparkRunner | 726.5 | 729.61 | 0.43 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_10_batch | RuntimeMs | SparkStructuredStreamingRunner | 1126.52 |
1211.0 | 7.5 | [[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_10_streaming | RuntimeMs | DataflowRunner | 105776.61 | 102899.66 |
-2.72 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_10_streaming | RuntimeMs | DirectRunner | 43926.21 | 44091.14 | 0.38 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_10_streaming | RuntimeMs | FlinkRunner | 1535.29 | 1488.14 | -3.07 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_11_batch | RuntimeMs | DataflowRunner | 39589.51 | 61325.57 | 54.9 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_11_batch | RuntimeMs | DirectRunner | 3757.79 | 3770.55 | 0.34 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_11_batch | RuntimeMs | FlinkRunner | 297.43 | 313.64 | 5.45 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_11_batch | RuntimeMs | SparkRunner | 1402.09 | 1399.52 | -0.18 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_11_batch | RuntimeMs | SparkStructuredStreamingRunner | 1135.95 |
1015.14 | -10.64 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_11_streaming | RuntimeMs | DataflowRunner | 182507.05 | 181193.81 |
-0.72 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_11_streaming | RuntimeMs | DirectRunner | 10696.0 | 10739.0 | 0.4 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_11_streaming | RuntimeMs | FlinkRunner | 1351.79 | 1319.05 | -2.42 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_12_batch | RuntimeMs | DataflowRunner | 19630.69 | 19666.75 | 0.18 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_12_batch | RuntimeMs | DirectRunner | 2597.07 | 2626.36 | 1.13 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_12_batch | RuntimeMs | FlinkRunner | 180.29 | 173.05 | -4.02 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_12_batch | RuntimeMs | SparkRunner | 924.52 | 929.25 | 0.51 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_12_batch | RuntimeMs | SparkStructuredStreamingRunner | 827.98 |
899.98 | 8.7 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_12_streaming | RuntimeMs | DataflowRunner | 96217.51 | 94780.91 |
-1.49 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_12_streaming | RuntimeMs | DirectRunner | 9335.68 | 9409.5 | 0.79 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_12_streaming | RuntimeMs | FlinkRunner | 621.5 | 620.73 | -0.12 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_13_batch | RuntimeMs | DataflowRunner | 33870.18 | 33840.36 | -0.09 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_13_batch | RuntimeMs | DirectRunner | 3225.36 | 3231.86 | 0.2 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_13_batch | RuntimeMs | FlinkRunner | 392.68 | 404.95 | 3.13 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_13_batch | RuntimeMs | SparkRunner | 355.27 | 357.02 | 0.49 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_13_batch | RuntimeMs | SparkStructuredStreamingRunner | 547.32 |
556.57 | 1.69 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_13_sql_batch | RuntimeMs | DataflowRunner | 62806.75 | 61308.68 |
-2.39 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_13_sql_batch | RuntimeMs | DirectRunner | 9065.15 | 9052.5 | -0.14 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_13_sql_batch | RuntimeMs | SparkRunner | 3473.45 | 3430.11 | -1.25 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_13_sql_batch | RuntimeMs | SparkStructuredStreamingRunner | 3113.52 |
3060.59 | -1.7 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_13_sql_streaming | RuntimeMs | DataflowRunner | 42997.25 | 34577.23 |
-19.58 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_13_sql_streaming | RuntimeMs | DirectRunner | 18918.29 | 18631.09 |
-1.52 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_13_sql_streaming | RuntimeMs | FlinkRunner | 28617.96 | 29781.77 |
4.07 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_13_streaming | RuntimeMs | DataflowRunner | 28812.61 | 24063.0 |
-16.48 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_13_streaming | RuntimeMs | DirectRunner | 9996.14 | 9949.18 | -0.47 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_13_streaming | RuntimeMs | FlinkRunner | 27765.36 | 28396.95 | 2.27 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_13_zetasql_batch | RuntimeMs | DataflowRunner | 262683.96 | 259338.23
| -1.27 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_13_zetasql_batch | RuntimeMs | DirectRunner | 10046.04 | 10059.45 |
0.13 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_13_zetasql_streaming | RuntimeMs | DataflowRunner | 116689.18 |
109084.86 | -6.52 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_13_zetasql_streaming | RuntimeMs | DirectRunner | 21036.07 | 21342.05
| 1.45 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_13_zetasql_streaming | RuntimeMs | FlinkRunner | 34089.0 | 34659.64 |
1.67 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_14_batch | RuntimeMs | DataflowRunner | 62350.36 | 61308.36 | -1.67 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_14_batch | RuntimeMs | DirectRunner | 4153.39 | 4150.05 | -0.08 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_14_batch | RuntimeMs | FlinkRunner | 666.86 | 650.5 | -2.45 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_14_batch | RuntimeMs | SparkRunner | 895.64 | 945.5 | 5.57 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_14_batch | RuntimeMs | SparkStructuredStreamingRunner | 933.43 |
941.27 | 0.84 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_14_streaming | RuntimeMs | DataflowRunner | 121083.79 | 126024.55 |
4.08 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_14_streaming | RuntimeMs | DirectRunner | 12338.79 | 12202.36 | -1.11
| [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_14_streaming | RuntimeMs | FlinkRunner | 1783.46 | 1762.27 | -1.19 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_15_batch | RuntimeMs | DataflowRunner | 45464.97 | 45823.83 | 0.79 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_15_batch | RuntimeMs | DirectRunner | 15421.04 | 15413.5 | -0.05 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_15_batch | RuntimeMs | FlinkRunner | 646.39 | 579.05 | -10.42 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_15_batch | RuntimeMs | SparkRunner | 1643.96 | 1675.16 | 1.9 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_15_batch | RuntimeMs | SparkStructuredStreamingRunner | 1451.93 |
1442.11 | -0.68 | [[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_16_batch | RuntimeMs | DataflowRunner | 51963.45 | 51419.82 | -1.05 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_16_batch | RuntimeMs | DirectRunner | 85652.86 | 84922.68 | -0.85 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_16_batch | RuntimeMs | FlinkRunner | 214.46 | 256.23 | 19.47 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_16_batch | RuntimeMs | SparkRunner | 1484.05 | 1386.3 | -6.59 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_16_batch | RuntimeMs | SparkStructuredStreamingRunner | 1249.75 |
1199.98 | -3.98 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_16_streaming | RuntimeMs | DataflowRunner | 89716.22 | 95450.93 | 6.39
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_16_streaming | RuntimeMs | DirectRunner | 124442.11 | 129482.0 | 4.05
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_16_streaming | RuntimeMs | FlinkRunner | 423.43 | 431.27 | 1.85 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_1_batch | RuntimeMs | DataflowRunner | 21877.32 | 20846.38 | -4.71 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_1_batch | RuntimeMs | DirectRunner | 3502.18 | 3394.41 | -3.08 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_1_batch | RuntimeMs | FlinkRunner | 315.89 | 307.91 | -2.53 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_1_batch | RuntimeMs | SparkRunner | 440.91 | 442.09 | 0.27 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_1_batch | RuntimeMs | SparkStructuredStreamingRunner | 591.54 | 592.73
| 0.2 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_1_sql_batch | RuntimeMs | DataflowRunner | 32344.21 | 32125.86 | -0.68
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_1_sql_batch | RuntimeMs | DirectRunner | 3546.3 | 3726.64 | 5.09 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_1_sql_batch | RuntimeMs | SparkRunner | 2430.38 | 2390.05 | -1.66 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_1_sql_batch | RuntimeMs | SparkStructuredStreamingRunner | 1572.41 |
1666.39 | 5.98 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_1_sql_streaming | RuntimeMs | DataflowRunner | 20870.43 | 21097.45 |
1.09 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_1_sql_streaming | RuntimeMs | DirectRunner | 11466.04 | 11491.86 |
0.23 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_1_sql_streaming | RuntimeMs | FlinkRunner | 667.54 | 637.77 | -4.46 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_1_streaming | RuntimeMs | DataflowRunner | 28937.91 | 28601.91 | -1.16
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_1_streaming | RuntimeMs | DirectRunner | 8206.14 | 8198.05 | -0.1 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_1_streaming | RuntimeMs | FlinkRunner | 648.86 | 667.5 | 2.87 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_2_batch | RuntimeMs | DataflowRunner | 15263.96 | 15124.28 | -0.92 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_2_batch | RuntimeMs | DirectRunner | 1530.89 | 1605.64 | 4.88 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_2_batch | RuntimeMs | FlinkRunner | 219.18 | 192.68 | -12.09 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_2_batch | RuntimeMs | SparkRunner | 311.59 | 310.82 | -0.25 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_2_batch | RuntimeMs | SparkStructuredStreamingRunner | 458.84 | 460.77
| 0.42 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_2_sql_batch | RuntimeMs | DataflowRunner | 25789.39 | 25085.68 | -2.73
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_2_sql_batch | RuntimeMs | DirectRunner | 1543.37 | 1555.64 | 0.79 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_2_sql_batch | RuntimeMs | SparkRunner | 744.41 | 744.02 | -0.05 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_2_sql_batch | RuntimeMs | SparkStructuredStreamingRunner | 1005.04 |
972.02 | -3.28 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_2_sql_streaming | RuntimeMs | DataflowRunner | 13841.29 | 13565.55 |
-1.99 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_2_sql_streaming | RuntimeMs | DirectRunner | 6789.04 | 6801.5 | 0.18 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_2_sql_streaming | RuntimeMs | FlinkRunner | 424.61 | 410.73 | -3.27 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_2_streaming | RuntimeMs | DataflowRunner | 19257.58 | 18899.86 | -1.86
| [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_2_streaming | RuntimeMs | DirectRunner | 5142.04 | 5144.05 | 0.04 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_2_streaming | RuntimeMs | FlinkRunner | 270.54 | 275.0 | 1.65 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_2_zetasql_batch | RuntimeMs | DataflowRunner | 71538.04 | 73956.5 |
3.38 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_2_zetasql_batch | RuntimeMs | DirectRunner | 2908.36 | 2883.55 | -0.85
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_2_zetasql_streaming | RuntimeMs | DataflowRunner | 39600.89 | 40732.09
| 2.86 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_2_zetasql_streaming | RuntimeMs | DirectRunner | 8218.75 | 8148.45 |
-0.86 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_2_zetasql_streaming | RuntimeMs | FlinkRunner | 1202.64 | 1165.27 |
-3.11 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_3_batch | RuntimeMs | DataflowRunner | 26748.14 | 27266.41 | 1.94 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_3_batch | RuntimeMs | DirectRunner | 3224.14 | 3243.5 | 0.6 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_3_batch | RuntimeMs | FlinkRunner | 1105.75 | 1059.41 | -4.19 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_3_batch | RuntimeMs | SparkRunner | 2086.2 | 2081.61 | -0.22 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_3_sql_batch | RuntimeMs | DataflowRunner | 29239.46 | 29789.05 | 1.88
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_3_sql_batch | RuntimeMs | DirectRunner | 3365.15 | 3329.5 | -1.06 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_3_sql_batch | RuntimeMs | SparkRunner | 3503.88 | 3480.3 | -0.67 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_3_sql_batch | RuntimeMs | SparkStructuredStreamingRunner | 3032.75 |
2996.09 | -1.21 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_3_sql_streaming | RuntimeMs | DataflowRunner | 33163.79 | 33241.18 |
0.23 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_3_sql_streaming | RuntimeMs | DirectRunner | 8460.71 | 8474.64 | 0.16
| [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_3_sql_streaming | RuntimeMs | FlinkRunner | 970.11 | 960.36 | -1.0 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_3_streaming | RuntimeMs | DataflowRunner | 43269.14 | 43554.49 | 0.66
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_3_streaming | RuntimeMs | DirectRunner | 7623.46 | 7534.73 | -1.16 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_3_streaming | RuntimeMs | FlinkRunner | 469.46 | 456.77 | -2.7 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_3_zetasql_batch | RuntimeMs | DataflowRunner | 54330.46 | 54539.23 |
0.38 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_3_zetasql_batch | RuntimeMs | DirectRunner | 3749.36 | 3745.59 | -0.1
| [[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_3_zetasql_streaming | RuntimeMs | DataflowRunner | 46361.32 | 46243.59
| -0.25 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_3_zetasql_streaming | RuntimeMs | DirectRunner | 8915.93 | 8861.45 |
-0.61 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_3_zetasql_streaming | RuntimeMs | FlinkRunner | 1191.43 | 1197.64 |
0.52 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_4_batch | RuntimeMs | DataflowRunner | 33212.25 | 33292.5 | 0.24 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_4_batch | RuntimeMs | DirectRunner | 2416.46 | 2409.05 | -0.31 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_4_batch | RuntimeMs | FlinkRunner | 861.68 | 851.14 | -1.22 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_4_batch | RuntimeMs | SparkRunner | 1001.52 | 998.98 | -0.25 |
[[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_4_batch | RuntimeMs | SparkStructuredStreamingRunner | 2243.59 |
2233.52 | -0.45 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_4_streaming | RuntimeMs | DataflowRunner | 49175.82 | 49133.95 | -0.09
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_4_streaming | RuntimeMs | DirectRunner | 2827.5 | 2862.05 | 1.22 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_4_streaming | RuntimeMs | FlinkRunner | 975.32 | 1021.45 | 4.73 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_5_batch | RuntimeMs | DataflowRunner | 35151.96 | 34758.3 | -1.12 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_5_batch | RuntimeMs | DirectRunner | 10582.36 | 10621.95 | 0.37 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_5_batch | RuntimeMs | FlinkRunner | 775.96 | 769.41 | -0.84 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_5_batch | RuntimeMs | SparkRunner | 1515.27 | 1535.14 | 1.31 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_5_batch | RuntimeMs | SparkStructuredStreamingRunner | 1719.73 |
1732.2 | 0.73 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_5_streaming | RuntimeMs | DataflowRunner | 214447.03 | 209021.26 |
-2.53 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_5_streaming | RuntimeMs | DirectRunner | 20728.25 | 21227.32 | 2.41 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_5_streaming | RuntimeMs | FlinkRunner | 1609.86 | 1637.59 | 1.72 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_6_batch | RuntimeMs | DataflowRunner | 31298.14 | 31001.5 | -0.95 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_6_batch | RuntimeMs | DirectRunner | 1886.82 | 1851.91 | -1.85 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_6_batch | RuntimeMs | FlinkRunner | 536.79 | 519.91 | -3.14 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_6_batch | RuntimeMs | SparkRunner | 710.09 | 706.14 | -0.56 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_6_batch | RuntimeMs | SparkStructuredStreamingRunner | 907.57 | 948.52
| 4.51 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_6_streaming | RuntimeMs | DataflowRunner | 38287.43 | 41809.0 | 9.2 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_6_streaming | RuntimeMs | DirectRunner | 2135.57 | 2098.95 | -1.71 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_6_streaming | RuntimeMs | FlinkRunner | 601.57 | 592.18 | -1.56 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_7_batch | RuntimeMs | DataflowRunner | 54975.39 | 56478.73 | 2.73 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_7_batch | RuntimeMs | DirectRunner | 18420.82 | 18547.23 | 0.69 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_7_batch | RuntimeMs | FlinkRunner | 665.07 | 677.59 | 1.88 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_7_batch | RuntimeMs | SparkRunner | 1440.04 | 1426.75 | -0.92 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_7_batch | RuntimeMs | SparkStructuredStreamingRunner | 2214.61 |
2196.27 | -0.83 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_7_sql_batch | RuntimeMs | DataflowRunner | 78570.29 | 78734.55 | 0.21
| [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_7_sql_batch | RuntimeMs | DirectRunner | 326788.15 | 364568.64 | 11.56
| [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_7_sql_batch | RuntimeMs | SparkRunner | 7212.82 | 7068.95 | -1.99 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_7_sql_batch | RuntimeMs | SparkStructuredStreamingRunner | 5996.48 |
5910.11 | -1.44 | [[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_7_sql_streaming | RuntimeMs | DataflowRunner | 1003130.36 | 1027200.23
| 2.4 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_7_sql_streaming | RuntimeMs | DirectRunner | 1076402.96 | 1215625.41 |
12.93 | [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_7_sql_streaming | RuntimeMs | FlinkRunner | 3486.11 | 3459.86 | -0.75
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_7_streaming | RuntimeMs | DataflowRunner | 120673.19 | 122350.28 |
1.39 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_7_streaming | RuntimeMs | DirectRunner | 850907.18 | 848503.82 | -0.28
| [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_7_streaming | RuntimeMs | FlinkRunner | 13410.82 | 13769.05 | 2.67 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_8_batch | RuntimeMs | DataflowRunner | 21572.36 | 21636.01 | 0.3 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_8_batch | RuntimeMs | DirectRunner | 2258.14 | 2383.27 | 5.54 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_8_batch | RuntimeMs | FlinkRunner | 473.39 | 478.09 | 0.99 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_8_batch | RuntimeMs | SparkRunner | 1451.93 | 1434.59 | -1.19 |
[[1]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_8_batch | RuntimeMs | SparkStructuredStreamingRunner | 1185.75 |
1128.57 | -4.82 | [[1]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_8_streaming | RuntimeMs | DataflowRunner | 59645.41 | 61147.95 | 2.52
| [[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_8_streaming | RuntimeMs | DirectRunner | 6981.75 | 7187.59 | 2.95 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_8_streaming | RuntimeMs | FlinkRunner | 479.46 | 478.59 | -0.18 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
nexmark_9_batch | RuntimeMs | DataflowRunner | 29188.29 | 30279.18 | 3.74 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[3]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_9_batch | RuntimeMs | DirectRunner | 1454.29 | 1440.0 | -0.98 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
nexmark_9_batch | RuntimeMs | FlinkRunner | 290.18 | 325.45 | 12.16 |
[[1]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_9_batch | RuntimeMs | SparkRunner | 541.73 | 544.66 | 0.54 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/tkqc0AdGk)
[[4]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_9_batch | RuntimeMs | SparkStructuredStreamingRunner | 708.32 | 716.98
| 1.22 | [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/f0Q6T0dGk)
[[3]](http://metrics.beam.apache.org/d/8INnSY9Mz)
[[4]](http://metrics.beam.apache.org/d/tkqc0AdGk)
nexmark_9_streaming | RuntimeMs | DataflowRunner | 31862.82 | 31803.18 | -0.19
| [[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_9_streaming | RuntimeMs | DirectRunner | 1758.86 | 1774.09 | 0.87 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
nexmark_9_streaming | RuntimeMs | FlinkRunner | 430.46 | 487.23 | 13.19 |
[[1]](http://metrics.beam.apache.org/d/ahudA_zGz)
[[2]](http://metrics.beam.apache.org/d/8INnSY9Mz)
Re: Beam Metrics Report (2022-07-12)
Posted by Danny McCormick via dev <de...@beam.apache.org>.
> Since we moved to Grafana for dashboards, I’m wondering if we can achieve
the similar things, that you proposed, with Grafana alerts mechanism.
Though, I guess it won’t be evident to find a commit that potentially
caused a performance degradation.
That's a good idea - I don't have enough context on what that offers to
speak too intelligently about it, but it would be nice if it could alert us
after 5 consecutive runs with degraded performance or something like that
(that definitely seems like the type of thing that is supported). It almost
certainly wouldn't have a way to support everything I described, but not
having to build and maintain our own scripts to handle it would probably be
worth that tradeoff.
On Thu, Jul 14, 2022 at 2:52 PM Alexey Romanenko <ar...@gmail.com>
wrote:
> Thanks Danny for your thoughts, I totally agree with you on a short and a
> long term.
>
> IINM, this report is created and sent via “report_generator.py" script
> that reads metrics from InfluxDB.
>
> Since we moved to Grafana for dashboards, I’m wondering if we can achieve
> the similar things, that you proposed, with Grafana alerts mechanism.
> Though, I guess it won’t be evident to find a commit that potentially
> caused a performance degradation.
>
> —
> Alexey
>
> On 14 Jul 2022, at 16:43, Danny McCormick via dev <de...@beam.apache.org>
> wrote:
>
> I like the idea of doing something like that, but using the median does
> mean we're likely going to be slower to detect regressions - for example,
> if we have an uptick on the 4th-7th day of the week, the median will not
> normally be affected. I would prefer something like excluding the top and
> bottom 5% of data points and then taking the mean. With that said, I think
> median is better than the current simple mean approach, so if that's an
> easier step I support it.
>
> In general, it would be awesome if we could make these more actionable (or
> replace them with something more actionable). My north star would be, for
> each performance test run:
>
> 1) Detect if performance has improved or degraded by more than ~1.5
> standard deviations from the median of the last 10 week.
> 2) If so, do several sample runs at the old commit and new commit to see
> if the regression is real.
>
> That would give us a commit range where a regression was introduced, which
> we could alert on. If we wanted, we could even take it a step further and:
>
> 3) Automatically do a git bisect to find the commit where performance
> degraded.
>
> That's longer term and should not block us from making shorter term
> improvements like what you're describing.
>
> Thanks,
> Danny
>
> On Thu, Jul 14, 2022 at 7:16 AM Alexey Romanenko <ar...@gmail.com>
> wrote:
>
>> Taking a look on red possible regressions number, I notice that it can be
>> caused by some outliers that may be happened because of, for example,
>> resources issues on Jenkins.
>>
>> <Screenshot 2022-07-14 at 13.08.14.png>
>>
>> (nexmark_11_batch, DataflowRunner)
>>
>>
>> Should we use a median value instead of a mean one to make it more
>> representative? Thoughts?
>>
>> —
>> Alexey
>>
>> On 12 Jul 2022, at 13:55, Apache Jenkins Server <
>> jenkins@builds.apache.org> wrote:
>>
>> Beam Metrics Report Color legend:
>> >= 20 >= 10 <= -10 <= -20 Possible regression
>> *Measurement* *Metric* *Runner* *Mean previous week* *Mean last week* *Diff
>> %* *Dashboard*
>> java_batch_cogbk_3 dataflow_v2_java17_runtime_sec - 13.58 16.39 20.7 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_4 dataflow_v2_java11_runtime_sec - 14.25 15.78 10.76 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_gbk_3 dataflow_v2_java11_runtime_sec - 14.44 17.31 19.88 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_v2_java17_runtime_sec - 35.86 39.61 10.45 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 sparkstructuredstreaming_runtime_sec - 84.12 92.67 10.17
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_pardo_1 dataflow_v2_java11_runtime_sec - 13.25 16.08 21.34 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_1 dataflow_v2_java17_runtime_sec - 15.09 16.7 10.66 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_2 dataflow_v2_java11_runtime_sec - 63.83 71.69 12.31 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_v2_java11_runtime_sec - 9.09 15.75 73.32 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_v2_java17_runtime_sec - 12.12 14.5 19.67 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_gbk_3 dataflow_v2_java11_runtime_sec - 53.16 59.27 11.48
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_pardo_3 dataflow_v2_java11_runtime_sec - 21.3 23.56 10.62
>> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> kafkaioit_results_runner_v2 read_time - 1.87 2.36 26.33 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> python_batch_cogbk_2 python_dataflow_batch_cogbk_2_runtime - 287.43 317.2
>> 10.36 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_gbk_3 python_dataflow_batch_gbk_3_runtime - 27.43 32.6 18.85
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_streaming_pardo_4 python_dataflow_streaming_pardo_4_runtime -
>> 538.57 602.0 11.78 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> nexmark_11_batch RuntimeMs DataflowRunner 39589.51 61325.57 54.9 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_16_batch RuntimeMs FlinkRunner 214.46 256.23 19.47 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_7_sql_batch RuntimeMs DirectRunner 326788.15 364568.64 11.56 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_sql_streaming RuntimeMs DirectRunner 1076402.96 1215625.41
>> 12.93 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_9_batch RuntimeMs FlinkRunner 290.18 325.45 12.16 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_9_streaming RuntimeMs FlinkRunner 430.46 487.23 13.19 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> Possible improvement
>> *Measurement* *Metric* *Runner* *Mean previous week* *Mean last week* *Diff
>> %* *Dashboard*
>> go_batch_gbk_7 dataflow_runtime - 125.77 111.91 -11.02 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_pardo_2 dataflow_runtime - 420.66 360.73 -14.25 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_cogbk_2 dataflow_v2_java11_runtime_sec - 50.65 44.37 -12.41
>> [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_gbk_3 dataflow_runtime_sec - 11.22 10.07 -10.3 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_6 dataflow_v2_java17_runtime_sec - 31.33 26.3 -16.06 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_runtime_sec - 49.25 44.03 -10.62 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_v2_java11_runtime_sec - 42.23 34.58 -18.11 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_pardo_1 sparkstructuredstreaming_runtime_sec - 56.17 49.55
>> -11.8 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_runtime_sec - 13.8 12.29 -10.98 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_cogbk_1 dataflow_v2_java11_runtime_sec - 3095.03 2628.35
>> -15.08 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_gbk_1 dataflow_v2_java17_runtime_sec - 1003.04 897.89
>> -10.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_2 dataflow_v2_java11_runtime_sec - 4656.02 2798.25
>> -39.9 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_3 dataflow_v2_java17_runtime_sec - 69.2 53.75 -22.32
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_5 dataflow_v2_java17_runtime_sec - 677.63 555.04
>> -18.09 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_pardo_1 dataflow_v2_java11_runtime_sec - 28.86 23.28
>> -19.35 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_4 dataflow_v2_java17_runtime_sec - 37.98 33.86
>> -10.85 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> kafkaioit_results read_time - 777.55 0.0 -100.0 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results write_time - 212.65 14.37 -93.24 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results_sdf_wrapper write_time - 6.25 3.97 -36.43 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> python_batch_cogbk_1 python_dataflow_batch_cogbk_1_runtime - 525.29 453.8
>> -13.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_cogbk_3 python_dataflow_batch_cogbk_3_runtime - 282.71 253.8
>> -10.23 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_combine_4 python_dataflow_batch_combine_4_runtime - 61.86
>> 53.0 -14.32 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> python_batch_sideinput_2 python_dataflow_batch_sideinput_2_runtime -
>> 50.14 45.0 -10.26 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_6 python_dataflow_batch_sideinput_6_runtime -
>> 45.29 38.0 -16.09 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_bqio_write_10GB_results bqio_write_10GB_results_runtime - 247.86
>> 210.2 -15.19 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
>> python_streaming_gbk_3 python_dataflow_streaming_gbk_3_runtime - 90.86
>> 78.5 -13.6 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> nexmark_11_batch RuntimeMs SparkStructuredStreamingRunner 1135.95 1015.14
>> -10.64 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_sql_streaming RuntimeMs DataflowRunner 42997.25 34577.23
>> -19.58 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_streaming RuntimeMs DataflowRunner 28812.61 24063.0 -16.48 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_15_batch RuntimeMs FlinkRunner 646.39 579.05 -10.42 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_2_batch RuntimeMs FlinkRunner 219.18 192.68 -12.09 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> All metrics
>> *Measurement* *Metric* *Runner* *Mean previous week* *Mean last week* *Diff
>> %* *Dashboard*
>> avroioit_hdfs_results read_time - 88.5 90.3 2.04 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> avroioit_hdfs_results write_time - 117.12 121.15 3.43 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> avroioit_results read_time - 90.57 93.45 3.18 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> avroioit_results write_time - 116.22 119.98 3.23 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> bqio_10GB_results_java_batch_avro avro_write_time - 35.46 34.18 -3.61 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> bqio_10GB_results_java_batch_avro read_time - 41.22 41.64 1.02 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> bqio_10GB_results_java_batch_json read_time - 42.36 38.94 -8.07 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> bqio_10GB_results_java_batch_json write_time - 51.59 50.42 -2.26 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> compressed_textioit_hdfs_results read_time - 158.32 160.91 1.64 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> compressed_textioit_hdfs_results write_time - 207.37 207.08 -0.14 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> compressed_textioit_results read_time - 158.41 161.48 1.94 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> compressed_textioit_results write_time - 204.95 204.34 -0.3 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> go_batch_cogbk_1 dataflow_runtime - 256.19 251.49 -1.84 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> go_batch_cogbk_2 dataflow_runtime - 148.89 143.6 -3.55 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> go_batch_cogbk_3 dataflow_runtime - 115.23 112.94 -1.99 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> go_batch_cogbk_4 dataflow_runtime - 114.24 112.66 -1.38 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> go_batch_combine_1 dataflow_runtime - 388.21 377.26 -2.82 [1]
>> <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> go_batch_combine_4 dataflow_runtime - 77.88 76.26 -2.09 [1]
>> <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> go_batch_combine_5 dataflow_runtime - 72.59 71.3 -1.78 [1]
>> <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> go_batch_gbk_1 dataflow_runtime - 371.28 369.86 -0.38 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_2 dataflow_runtime - 100.97 98.34 -2.6 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_3 dataflow_runtime - 31.82 30.17 -5.18 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_4 dataflow_runtime - 55.12 54.68 -0.78 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_5 dataflow_runtime - 45.18 45.82 1.42 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_6 dataflow_runtime - 102.59 103.03 0.44 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_7 dataflow_runtime - 125.77 111.91 -11.02 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_pardo_1 dataflow_runtime - 75.89 78.22 3.07 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> go_batch_pardo_2 dataflow_runtime - 420.66 360.73 -14.25 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> go_batch_pardo_3 dataflow_runtime - 63.86 65.18 2.07 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> go_batch_pardo_4 dataflow_runtime - 72.36 70.43 -2.67 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> go_batch_sideinput_3 dataflow_runtime - 2.35 2.44 3.86 [1]
>> <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> go_batch_sideinput_4 dataflow_runtime - 208.13 206.1 -0.98 [1]
>> <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> hadoopformatioit_results read_time - 6.33 6.34 0.13 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> hadoopformatioit_results write_time - 7.79 7.51 -3.63 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> java_batch_cogbk_1 dataflow_runtime_sec - 41.75 42.28 1.27 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_1 dataflow_v2_java11_runtime_sec - 46.39 43.44 -6.36 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_1 dataflow_v2_java17_runtime_sec - 49.57 45.83 -7.54 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_1 sparkstructuredstreaming_runtime_sec - 118.05 124.41
>> 5.39 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_2 dataflow_runtime_sec - 39.63 42.27 6.67 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_2 dataflow_v2_java11_runtime_sec - 50.65 44.37 -12.41
>> [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_2 dataflow_v2_java17_runtime_sec - 44.58 44.66 0.18 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_2 sparkstructuredstreaming_runtime_sec - 116.79 122.12
>> 4.56 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_3 dataflow_runtime_sec - 15.17 15.82 4.32 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_3 dataflow_v2_java11_runtime_sec - 14.41 13.74 -4.68 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_3 dataflow_v2_java17_runtime_sec - 13.58 16.39 20.7 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_3 sparkstructuredstreaming_runtime_sec - 19.96 20.61
>> 3.28 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_4 dataflow_runtime_sec - 13.12 13.65 4.05 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_4 dataflow_v2_java11_runtime_sec - 14.25 15.78 10.76 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_4 dataflow_v2_java17_runtime_sec - 13.23 13.71 3.59 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_4 sparkstructuredstreaming_runtime_sec - 20.37 20.12
>> -1.19 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_combine_1 dataflow_runtime_sec - 64.0 62.62 -2.16 [1]
>> <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_combine_1 sparkstructuredstreaming_runtime_sec - 554.59 557.39
>> 0.5 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_combine_4 dataflow_runtime_sec - 27.99 27.76 -0.8 [1]
>> <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_combine_4 sparkstructuredstreaming_runtime_sec - 124.67 126.83
>> 1.73 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_combine_5 dataflow_runtime_sec - 27.27 24.84 -8.93 [1]
>> <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_combine_5 sparkstructuredstreaming_runtime_sec - 119.92 120.06
>> 0.12 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_gbk_1 dataflow_runtime_sec - 125.42 115.83 -7.64 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_1 dataflow_v2_java11_runtime_sec - 96.68 95.35 -1.38 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_1 dataflow_v2_java17_runtime_sec - 86.51 90.35 4.44 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_1 sparkstructuredstreaming_runtime_sec - 502.91 503.81
>> 0.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_2 dataflow_runtime_sec - 33.68 32.33 -4.0 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_2 dataflow_v2_java11_runtime_sec - 42.27 40.93 -3.18 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_2 dataflow_v2_java17_runtime_sec - 39.31 40.94 4.12 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_2 sparkstructuredstreaming_runtime_sec - 97.8 107.11 9.52
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_3 dataflow_runtime_sec - 11.22 10.07 -10.3 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_3 dataflow_v2_java11_runtime_sec - 14.44 17.31 19.88 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_3 dataflow_v2_java17_runtime_sec - 15.28 15.39 0.75 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_3 sparkstructuredstreaming_runtime_sec - 8.2 8.74 6.56 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_4 dataflow_runtime_sec - 18.85 19.23 2.05 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_4 dataflow_v2_java11_runtime_sec - 22.58 22.18 -1.78 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_4 dataflow_v2_java17_runtime_sec - 23.02 24.62 6.92 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_4 sparkstructuredstreaming_runtime_sec - 97.45 101.76 4.43
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_5 dataflow_runtime_sec - 18.5 17.83 -3.64 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_5 dataflow_v2_java11_runtime_sec - 17.53 17.45 -0.49 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_5 dataflow_v2_java17_runtime_sec - 19.43 20.54 5.68 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_5 sparkstructuredstreaming_runtime_sec - 100.52 100.52
>> -0.0 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_6 dataflow_runtime_sec - 37.2 34.88 -6.26 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_6 dataflow_v2_java11_runtime_sec - 26.4 26.36 -0.12 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_6 dataflow_v2_java17_runtime_sec - 31.33 26.3 -16.06 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_6 sparkstructuredstreaming_runtime_sec - 58.34 60.86 4.32
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_runtime_sec - 49.25 44.03 -10.62 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_v2_java11_runtime_sec - 42.23 34.58 -18.11 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_v2_java17_runtime_sec - 35.86 39.61 10.45 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 sparkstructuredstreaming_runtime_sec - 84.12 92.67 10.17
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_pardo_1 dataflow_runtime_sec - 20.04 18.14 -9.49 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_1 dataflow_v2_java11_runtime_sec - 13.25 16.08 21.34 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_1 dataflow_v2_java17_runtime_sec - 15.09 16.7 10.66 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_1 sparkstructuredstreaming_runtime_sec - 56.17 49.55
>> -11.8 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_2 dataflow_runtime_sec - 186.08 202.59 8.87 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_2 dataflow_v2_java11_runtime_sec - 63.83 71.69 12.31 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_2 dataflow_v2_java17_runtime_sec - 66.38 67.13 1.13 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_runtime_sec - 13.8 12.29 -10.98 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_v2_java11_runtime_sec - 9.09 15.75 73.32 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_v2_java17_runtime_sec - 12.12 14.5 19.67 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_4 dataflow_runtime_sec - 24.8 22.52 -9.18 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_4 dataflow_v2_java11_runtime_sec - 26.58 24.46 -7.98 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_4 dataflow_v2_java17_runtime_sec - 25.41 24.93 -1.9 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_cogbk_1 dataflow_runtime_sec - 927.84 858.74 -7.45 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_1 dataflow_v2_java11_runtime_sec - 3095.03 2628.35
>> -15.08 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_1 dataflow_v2_java17_runtime_sec - 2589.29 2595.64
>> 0.25 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_2 dataflow_runtime_sec - 907.41 837.58 -7.7 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_2 dataflow_v2_java11_runtime_sec - 3256.56 2969.53
>> -8.81 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_2 dataflow_v2_java17_runtime_sec - 2938.01 3090.66
>> 5.2 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_3 dataflow_runtime_sec - 118.04 122.42 3.71 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_3 dataflow_v2_java11_runtime_sec - 322.79 326.12
>> 1.03 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_3 dataflow_v2_java17_runtime_sec - 319.04 325.44
>> 2.01 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_4 dataflow_runtime_sec - 125.95 122.06 -3.09 [1]
>> <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_4 dataflow_v2_java11_runtime_sec - 320.92 326.47
>> 1.73 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_4 dataflow_v2_java17_runtime_sec - 331.9 322.66
>> -2.78 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_combine_1 dataflow_runtime_sec - 509.03 511.79 0.54 [1]
>> <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_streaming_combine_4 dataflow_runtime_sec - 318.94 322.22 1.03 [1]
>> <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_streaming_combine_5 dataflow_runtime_sec - 307.67 309.5 0.6 [1]
>> <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_streaming_gbk_1 dataflow_runtime_sec - 670.68 646.17 -3.65 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_1 dataflow_v2_java11_runtime_sec - 875.0 920.0 5.14
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_1 dataflow_v2_java17_runtime_sec - 1003.04 897.89
>> -10.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_2 dataflow_runtime_sec - 803.33 798.42 -0.61 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_2 dataflow_v2_java11_runtime_sec - 4656.02 2798.25
>> -39.9 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_2 dataflow_v2_java17_runtime_sec - 3158.86 3166.16
>> 0.23 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_3 dataflow_runtime_sec - 23.62 24.92 5.49 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_3 dataflow_v2_java11_runtime_sec - 53.16 59.27 11.48
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_3 dataflow_v2_java17_runtime_sec - 69.2 53.75 -22.32
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_4 dataflow_runtime_sec - 289.5 286.49 -1.04 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_4 dataflow_v2_java11_runtime_sec - 641.36 649.91 1.33
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_4 dataflow_v2_java17_runtime_sec - 665.68 685.61 2.99
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_5 dataflow_runtime_sec - 293.31 280.95 -4.21 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_5 dataflow_v2_java11_runtime_sec - 602.18 632.83 5.09
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_5 dataflow_v2_java17_runtime_sec - 677.63 555.04
>> -18.09 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_6 dataflow_runtime_sec - 73.86 76.09 3.02 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_6 dataflow_v2_java11_runtime_sec - 160.18 145.57 -9.12
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_6 dataflow_v2_java17_runtime_sec - 194.24 178.72 -7.99
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_7 dataflow_runtime_sec - 137.68 137.62 -0.04 [1]
>> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_7 dataflow_v2_java11_runtime_sec - 207.62 226.19 8.94
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_7 dataflow_v2_java17_runtime_sec - 223.14 202.22 -9.37
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_pardo_1 dataflow_runtime_sec - 18.79 18.93 0.75 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_1 dataflow_v2_java11_runtime_sec - 28.86 23.28
>> -19.35 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_1 dataflow_v2_java17_runtime_sec - 25.27 24.23 -4.08
>> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_2 dataflow_runtime_sec - 223.51 226.23 1.22 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_2 dataflow_v2_java11_runtime_sec - 114.31 111.55
>> -2.41 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_2 dataflow_v2_java17_runtime_sec - 105.51 97.51
>> -7.58 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_3 dataflow_runtime_sec - 12.55 12.71 1.25 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_3 dataflow_v2_java11_runtime_sec - 21.3 23.56 10.62
>> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_3 dataflow_v2_java17_runtime_sec - 24.03 24.17 0.61
>> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_4 dataflow_runtime_sec - 32.98 33.47 1.49 [1]
>> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_4 dataflow_v2_java11_runtime_sec - 36.78 35.46 -3.61
>> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_4 dataflow_v2_java17_runtime_sec - 37.98 33.86
>> -10.85 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> jdbcioit_results read_time - 24.89 24.82 -0.28 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> jdbcioit_results write_time - 38.16 36.52 -4.3 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results read_time - 777.55 0.0 -100.0 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results write_time - 212.65 14.37 -93.24 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results_runner_v2 read_time - 1.87 2.36 26.33 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results_runner_v2 write_time - 5.89 5.91 0.4 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results_sdf_wrapper read_time - 3.04 3.26 7.08 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results_sdf_wrapper write_time - 6.25 3.97 -36.43 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> many_files_textioit_hdfs_results read_time - 17.29 17.48 1.13 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> many_files_textioit_hdfs_results write_time - 61.16 56.82 -7.09 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> many_files_textioit_results read_time - 24.9 24.23 -2.67 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> many_files_textioit_results write_time - 61.41 58.16 -5.29 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> mongodbioit_results read_time - 30.18 30.83 2.15 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> mongodbioit_results write_time - 59.02 55.45 -6.06 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> parquetioit_hdfs_results read_time - 88.2 88.97 0.88 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> parquetioit_hdfs_results write_time - 147.1 142.12 -3.39 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> parquetioit_results read_time - 87.88 89.9 2.29 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> parquetioit_results write_time - 145.06 139.23 -4.02 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> python_batch_cogbk_1 python_dataflow_batch_cogbk_1_runtime - 525.29 453.8
>> -13.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_cogbk_2 python_dataflow_batch_cogbk_2_runtime - 287.43 317.2
>> 10.36 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_cogbk_3 python_dataflow_batch_cogbk_3_runtime - 282.71 253.8
>> -10.23 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_cogbk_4 python_dataflow_batch_cogbk_4_runtime - 291.71 278.4
>> -4.56 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_combine_1 python_dataflow_batch_combine_1_runtime - 841.29
>> 875.6 4.08 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> python_batch_combine_4 python_dataflow_batch_combine_4_runtime - 61.86
>> 53.0 -14.32 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> python_batch_combine_5 python_dataflow_batch_combine_5_runtime - 46.0
>> 44.4 -3.48 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> python_batch_gbk_1 python_dataflow_batch_gbk_1_runtime - 1877.29 1915.8
>> 2.05 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_2 python_dataflow_batch_gbk_2_runtime - 373.14 367.8
>> -1.43 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_3 python_dataflow_batch_gbk_3_runtime - 27.43 32.6 18.85
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_4 python_dataflow_batch_gbk_4_runtime - 124.43 124.2
>> -0.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_5 python_dataflow_batch_gbk_5_runtime - 127.0 129.4 1.89
>> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_6 python_dataflow_batch_gbk_6_runtime - 344.86 352.4
>> 2.19 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_7 python_dataflow_batch_gbk_7_runtime - 353.14 346.0
>> -2.02 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_pardo_1 python_dataflow_batch_pardo_1_runtime - 180.14 176.8
>> -1.86 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_batch_pardo_2 python_dataflow_batch_pardo_2_runtime - 1635.57
>> 1661.4 1.58 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_batch_pardo_3 python_dataflow_batch_pardo_3_runtime - 141.86 150.6
>> 6.16 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_batch_pardo_4 python_dataflow_batch_pardo_4_runtime - 326.71 345.2
>> 5.66 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_batch_sideinput_1 python_dataflow_batch_sideinput_1_runtime -
>> 47.86 46.4 -3.04 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_2 python_dataflow_batch_sideinput_2_runtime -
>> 50.14 45.0 -10.26 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_3 python_dataflow_batch_sideinput_3_runtime - 5.71
>> 5.4 -5.5 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_4 python_dataflow_batch_sideinput_4_runtime -
>> 471.29 436.4 -7.4 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_5 python_dataflow_batch_sideinput_5_runtime -
>> 47.29 44.4 -6.1 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_6 python_dataflow_batch_sideinput_6_runtime -
>> 45.29 38.0 -16.09 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_7 python_dataflow_batch_sideinput_7_runtime -
>> 5566.29 5588.2 0.39 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_8 python_dataflow_batch_sideinput_8_runtime -
>> 5592.14 5562.8 -0.52 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_bqio_read_10GB_results bqio_read_10GB_results_runtime - 118.71
>> 111.6 -5.99 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
>> python_bqio_write_10GB_results bqio_write_10GB_results_runtime - 247.86
>> 210.2 -15.19 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
>> python_direct_microbenchmarks fn_api_runner_microbenchmark_runtime_sec -
>> 94.83 94.11 -0.76
>> python_direct_microbenchmarks teststream_microbenchmark_runtime_sec -
>> 344.58 337.86 -1.95
>> python_spanner_write_2GB_results pyspanner_write_2GB_results_runtime -
>> 1562.57 1599.2 2.34 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
>> python_spannerio_read_2GB_results pyspanner_read_2GB_results_runtime -
>> 26.43 23.8 -9.95 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
>> python_streaming_cogbk_1 python_dataflow_streaming_cogbk_1_runtime -
>> 628.29 659.8 5.02 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_streaming_cogbk_2 python_dataflow_streaming_cogbk_2_runtime -
>> 425.0 422.4 -0.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_streaming_cogbk_3 python_dataflow_streaming_cogbk_3_runtime -
>> 506.43 491.2 -3.01 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_streaming_cogbk_4 python_dataflow_streaming_cogbk_4_runtime -
>> 424.71 427.0 0.54 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_streaming_gbk_3 python_dataflow_streaming_gbk_3_runtime - 90.86
>> 78.5 -13.6 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_streaming_gbk_6 python_dataflow_streaming_gbk_6_runtime - 523.57
>> 546.0 4.28 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_streaming_gbk_7 python_dataflow_streaming_gbk_7_runtime - 506.29
>> 553.5 9.33 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_streaming_pardo_1 python_dataflow_streaming_pardo_1_runtime -
>> 253.57 235.5 -7.13 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_streaming_pardo_2 python_dataflow_streaming_pardo_2_runtime -
>> 2941.57 3096.5 5.27 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_streaming_pardo_3 python_dataflow_streaming_pardo_3_runtime -
>> 204.0 200.0 -1.96 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_streaming_pardo_4 python_dataflow_streaming_pardo_4_runtime -
>> 538.57 602.0 11.78 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> textioit_hdfs_results read_time - 19.88 20.73 4.27 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> textioit_hdfs_results write_time - 32.34 31.18 -3.59 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> textioit_results read_time - 20.52 21.58 5.16 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> textioit_results write_time - 32.12 32.81 2.14 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> tfrecordioit_results read_time - 15.43 16.26 5.35 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> tfrecordioit_results write_time - 20.79 21.14 1.68 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> wordcount_py37_results runtime - 811.08 824.06 1.6 [1]
>> <http://metrics.beam.apache.org/d/1cnwVDkGk>
>> xmlioit_hdfs_results read_time - 21.4 22.77 6.44 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> xmlioit_hdfs_results write_time - 24.67 25.12 1.79 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> xmlioit_results read_time - 23.48 23.36 -0.53 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> xmlioit_results write_time - 24.44 26.24 7.36 [1]
>> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> nexmark_0_batch RuntimeMs DataflowRunner 27146.02 27212.08 0.24 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_batch RuntimeMs DirectRunner 4782.36 4783.0 0.01 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_batch RuntimeMs FlinkRunner 1155.07 1111.86 -3.74 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_0_batch RuntimeMs SparkRunner 1253.64 1255.18 0.12 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_batch RuntimeMs SparkStructuredStreamingRunner 1546.91 1554.18
>> 0.47 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_0_sql_batch RuntimeMs DataflowRunner 35274.21 35610.5 0.95 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_0_sql_batch RuntimeMs DirectRunner 5570.26 5655.59 1.53 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_sql_batch RuntimeMs SparkRunner 3205.2 3141.48 -1.99 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_0_sql_batch RuntimeMs SparkStructuredStreamingRunner 2939.46
>> 2926.11 -0.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_0_sql_streaming RuntimeMs DataflowRunner 24026.89 23121.55 -3.77
>> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_sql_streaming RuntimeMs DirectRunner 15501.0 15555.91 0.35 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_sql_streaming RuntimeMs FlinkRunner 1637.46 1628.64 -0.54 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_0_streaming RuntimeMs DataflowRunner 39291.92 37816.14 -3.76 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_streaming RuntimeMs DirectRunner 11811.79 11938.45 1.07 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_streaming RuntimeMs FlinkRunner 1432.14 1422.36 -0.68 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_0_zetasql_batch RuntimeMs DataflowRunner 114915.96 112238.45
>> -2.33 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_0_zetasql_batch RuntimeMs DirectRunner 6757.07 6834.27 1.14 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_0_zetasql_streaming RuntimeMs DataflowRunner 67751.04 66627.91
>> -1.66 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_zetasql_streaming RuntimeMs DirectRunner 17105.75 16996.45
>> -0.64 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_zetasql_streaming RuntimeMs FlinkRunner 3248.54 3261.41 0.4 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_10_batch RuntimeMs DataflowRunner 48690.3 49306.87 1.27 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_10_batch RuntimeMs DirectRunner 10463.32 10465.09 0.02 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_10_batch RuntimeMs FlinkRunner 777.21 714.64 -8.05 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_10_batch RuntimeMs SparkRunner 726.5 729.61 0.43 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_10_batch RuntimeMs SparkStructuredStreamingRunner 1126.52 1211.0
>> 7.5 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_10_streaming RuntimeMs DataflowRunner 105776.61 102899.66 -2.72
>> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_10_streaming RuntimeMs DirectRunner 43926.21 44091.14 0.38 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_10_streaming RuntimeMs FlinkRunner 1535.29 1488.14 -3.07 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_11_batch RuntimeMs DataflowRunner 39589.51 61325.57 54.9 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_11_batch RuntimeMs DirectRunner 3757.79 3770.55 0.34 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_11_batch RuntimeMs FlinkRunner 297.43 313.64 5.45 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_11_batch RuntimeMs SparkRunner 1402.09 1399.52 -0.18 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_11_batch RuntimeMs SparkStructuredStreamingRunner 1135.95 1015.14
>> -10.64 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_11_streaming RuntimeMs DataflowRunner 182507.05 181193.81 -0.72
>> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_11_streaming RuntimeMs DirectRunner 10696.0 10739.0 0.4 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_11_streaming RuntimeMs FlinkRunner 1351.79 1319.05 -2.42 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_12_batch RuntimeMs DataflowRunner 19630.69 19666.75 0.18 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_12_batch RuntimeMs DirectRunner 2597.07 2626.36 1.13 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_12_batch RuntimeMs FlinkRunner 180.29 173.05 -4.02 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_12_batch RuntimeMs SparkRunner 924.52 929.25 0.51 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_12_batch RuntimeMs SparkStructuredStreamingRunner 827.98 899.98
>> 8.7 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_12_streaming RuntimeMs DataflowRunner 96217.51 94780.91 -1.49 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_12_streaming RuntimeMs DirectRunner 9335.68 9409.5 0.79 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_12_streaming RuntimeMs FlinkRunner 621.5 620.73 -0.12 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_batch RuntimeMs DataflowRunner 33870.18 33840.36 -0.09 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_batch RuntimeMs DirectRunner 3225.36 3231.86 0.2 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_13_batch RuntimeMs FlinkRunner 392.68 404.95 3.13 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_13_batch RuntimeMs SparkRunner 355.27 357.02 0.49 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_batch RuntimeMs SparkStructuredStreamingRunner 547.32 556.57
>> 1.69 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_13_sql_batch RuntimeMs DataflowRunner 62806.75 61308.68 -2.39 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_13_sql_batch RuntimeMs DirectRunner 9065.15 9052.5 -0.14 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_sql_batch RuntimeMs SparkRunner 3473.45 3430.11 -1.25 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_sql_batch RuntimeMs SparkStructuredStreamingRunner 3113.52
>> 3060.59 -1.7 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_sql_streaming RuntimeMs DataflowRunner 42997.25 34577.23
>> -19.58 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_sql_streaming RuntimeMs DirectRunner 18918.29 18631.09 -1.52
>> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_sql_streaming RuntimeMs FlinkRunner 28617.96 29781.77 4.07 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_streaming RuntimeMs DataflowRunner 28812.61 24063.0 -16.48 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_streaming RuntimeMs DirectRunner 9996.14 9949.18 -0.47 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_streaming RuntimeMs FlinkRunner 27765.36 28396.95 2.27 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_zetasql_batch RuntimeMs DataflowRunner 262683.96 259338.23
>> -1.27 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_13_zetasql_batch RuntimeMs DirectRunner 10046.04 10059.45 0.13
>> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_zetasql_streaming RuntimeMs DataflowRunner 116689.18 109084.86
>> -6.52 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_zetasql_streaming RuntimeMs DirectRunner 21036.07 21342.05
>> 1.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_zetasql_streaming RuntimeMs FlinkRunner 34089.0 34659.64 1.67
>> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_14_batch RuntimeMs DataflowRunner 62350.36 61308.36 -1.67 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_14_batch RuntimeMs DirectRunner 4153.39 4150.05 -0.08 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_14_batch RuntimeMs FlinkRunner 666.86 650.5 -2.45 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_14_batch RuntimeMs SparkRunner 895.64 945.5 5.57 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_14_batch RuntimeMs SparkStructuredStreamingRunner 933.43 941.27
>> 0.84 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_14_streaming RuntimeMs DataflowRunner 121083.79 126024.55 4.08
>> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_14_streaming RuntimeMs DirectRunner 12338.79 12202.36 -1.11 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_14_streaming RuntimeMs FlinkRunner 1783.46 1762.27 -1.19 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_15_batch RuntimeMs DataflowRunner 45464.97 45823.83 0.79 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_15_batch RuntimeMs DirectRunner 15421.04 15413.5 -0.05 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_15_batch RuntimeMs FlinkRunner 646.39 579.05 -10.42 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_15_batch RuntimeMs SparkRunner 1643.96 1675.16 1.9 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_15_batch RuntimeMs SparkStructuredStreamingRunner 1451.93 1442.11
>> -0.68 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_16_batch RuntimeMs DataflowRunner 51963.45 51419.82 -1.05 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_16_batch RuntimeMs DirectRunner 85652.86 84922.68 -0.85 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_16_batch RuntimeMs FlinkRunner 214.46 256.23 19.47 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_16_batch RuntimeMs SparkRunner 1484.05 1386.3 -6.59 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_16_batch RuntimeMs SparkStructuredStreamingRunner 1249.75 1199.98
>> -3.98 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_16_streaming RuntimeMs DataflowRunner 89716.22 95450.93 6.39 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_16_streaming RuntimeMs DirectRunner 124442.11 129482.0 4.05 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_16_streaming RuntimeMs FlinkRunner 423.43 431.27 1.85 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_batch RuntimeMs DataflowRunner 21877.32 20846.38 -4.71 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_batch RuntimeMs DirectRunner 3502.18 3394.41 -3.08 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_1_batch RuntimeMs FlinkRunner 315.89 307.91 -2.53 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_1_batch RuntimeMs SparkRunner 440.91 442.09 0.27 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_1_batch RuntimeMs SparkStructuredStreamingRunner 591.54 592.73
>> 0.2 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_1_sql_batch RuntimeMs DataflowRunner 32344.21 32125.86 -0.68 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_1_sql_batch RuntimeMs DirectRunner 3546.3 3726.64 5.09 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_sql_batch RuntimeMs SparkRunner 2430.38 2390.05 -1.66 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_1_sql_batch RuntimeMs SparkStructuredStreamingRunner 1572.41
>> 1666.39 5.98 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_1_sql_streaming RuntimeMs DataflowRunner 20870.43 21097.45 1.09
>> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_1_sql_streaming RuntimeMs DirectRunner 11466.04 11491.86 0.23 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_sql_streaming RuntimeMs FlinkRunner 667.54 637.77 -4.46 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_streaming RuntimeMs DataflowRunner 28937.91 28601.91 -1.16 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_streaming RuntimeMs DirectRunner 8206.14 8198.05 -0.1 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_streaming RuntimeMs FlinkRunner 648.86 667.5 2.87 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_batch RuntimeMs DataflowRunner 15263.96 15124.28 -0.92 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_2_batch RuntimeMs DirectRunner 1530.89 1605.64 4.88 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_2_batch RuntimeMs FlinkRunner 219.18 192.68 -12.09 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_batch RuntimeMs SparkRunner 311.59 310.82 -0.25 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_batch RuntimeMs SparkStructuredStreamingRunner 458.84 460.77
>> 0.42 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_2_sql_batch RuntimeMs DataflowRunner 25789.39 25085.68 -2.73 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_2_sql_batch RuntimeMs DirectRunner 1543.37 1555.64 0.79 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_2_sql_batch RuntimeMs SparkRunner 744.41 744.02 -0.05 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_sql_batch RuntimeMs SparkStructuredStreamingRunner 1005.04
>> 972.02 -3.28 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_2_sql_streaming RuntimeMs DataflowRunner 13841.29 13565.55 -1.99
>> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_sql_streaming RuntimeMs DirectRunner 6789.04 6801.5 0.18 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_sql_streaming RuntimeMs FlinkRunner 424.61 410.73 -3.27 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_streaming RuntimeMs DataflowRunner 19257.58 18899.86 -1.86 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_streaming RuntimeMs DirectRunner 5142.04 5144.05 0.04 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_streaming RuntimeMs FlinkRunner 270.54 275.0 1.65 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_zetasql_batch RuntimeMs DataflowRunner 71538.04 73956.5 3.38
>> [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_zetasql_batch RuntimeMs DirectRunner 2908.36 2883.55 -0.85 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_zetasql_streaming RuntimeMs DataflowRunner 39600.89 40732.09
>> 2.86 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_zetasql_streaming RuntimeMs DirectRunner 8218.75 8148.45 -0.86
>> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_zetasql_streaming RuntimeMs FlinkRunner 1202.64 1165.27 -3.11
>> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_batch RuntimeMs DataflowRunner 26748.14 27266.41 1.94 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_batch RuntimeMs DirectRunner 3224.14 3243.5 0.6 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_batch RuntimeMs FlinkRunner 1105.75 1059.41 -4.19 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_3_batch RuntimeMs SparkRunner 2086.2 2081.61 -0.22 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_sql_batch RuntimeMs DataflowRunner 29239.46 29789.05 1.88 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_3_sql_batch RuntimeMs DirectRunner 3365.15 3329.5 -1.06 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_3_sql_batch RuntimeMs SparkRunner 3503.88 3480.3 -0.67 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_3_sql_batch RuntimeMs SparkStructuredStreamingRunner 3032.75
>> 2996.09 -1.21 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_3_sql_streaming RuntimeMs DataflowRunner 33163.79 33241.18 0.23
>> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_sql_streaming RuntimeMs DirectRunner 8460.71 8474.64 0.16 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_sql_streaming RuntimeMs FlinkRunner 970.11 960.36 -1.0 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_streaming RuntimeMs DataflowRunner 43269.14 43554.49 0.66 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_streaming RuntimeMs DirectRunner 7623.46 7534.73 -1.16 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_streaming RuntimeMs FlinkRunner 469.46 456.77 -2.7 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_zetasql_batch RuntimeMs DataflowRunner 54330.46 54539.23 0.38
>> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_3_zetasql_batch RuntimeMs DirectRunner 3749.36 3745.59 -0.1 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_zetasql_streaming RuntimeMs DataflowRunner 46361.32 46243.59
>> -0.25 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_zetasql_streaming RuntimeMs DirectRunner 8915.93 8861.45 -0.61
>> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_zetasql_streaming RuntimeMs FlinkRunner 1191.43 1197.64 0.52
>> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_4_batch RuntimeMs DataflowRunner 33212.25 33292.5 0.24 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_4_batch RuntimeMs DirectRunner 2416.46 2409.05 -0.31 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_4_batch RuntimeMs FlinkRunner 861.68 851.14 -1.22 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_4_batch RuntimeMs SparkRunner 1001.52 998.98 -0.25 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_4_batch RuntimeMs SparkStructuredStreamingRunner 2243.59 2233.52
>> -0.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_4_streaming RuntimeMs DataflowRunner 49175.82 49133.95 -0.09 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_4_streaming RuntimeMs DirectRunner 2827.5 2862.05 1.22 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_4_streaming RuntimeMs FlinkRunner 975.32 1021.45 4.73 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_5_batch RuntimeMs DataflowRunner 35151.96 34758.3 -1.12 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_5_batch RuntimeMs DirectRunner 10582.36 10621.95 0.37 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_5_batch RuntimeMs FlinkRunner 775.96 769.41 -0.84 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_5_batch RuntimeMs SparkRunner 1515.27 1535.14 1.31 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_5_batch RuntimeMs SparkStructuredStreamingRunner 1719.73 1732.2
>> 0.73 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_5_streaming RuntimeMs DataflowRunner 214447.03 209021.26 -2.53
>> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_5_streaming RuntimeMs DirectRunner 20728.25 21227.32 2.41 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_5_streaming RuntimeMs FlinkRunner 1609.86 1637.59 1.72 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_6_batch RuntimeMs DataflowRunner 31298.14 31001.5 -0.95 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_6_batch RuntimeMs DirectRunner 1886.82 1851.91 -1.85 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_6_batch RuntimeMs FlinkRunner 536.79 519.91 -3.14 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_6_batch RuntimeMs SparkRunner 710.09 706.14 -0.56 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_6_batch RuntimeMs SparkStructuredStreamingRunner 907.57 948.52
>> 4.51 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_6_streaming RuntimeMs DataflowRunner 38287.43 41809.0 9.2 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_6_streaming RuntimeMs DirectRunner 2135.57 2098.95 -1.71 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_6_streaming RuntimeMs FlinkRunner 601.57 592.18 -1.56 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_batch RuntimeMs DataflowRunner 54975.39 56478.73 2.73 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_7_batch RuntimeMs DirectRunner 18420.82 18547.23 0.69 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_7_batch RuntimeMs FlinkRunner 665.07 677.59 1.88 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_7_batch RuntimeMs SparkRunner 1440.04 1426.75 -0.92 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_7_batch RuntimeMs SparkStructuredStreamingRunner 2214.61 2196.27
>> -0.83 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_7_sql_batch RuntimeMs DataflowRunner 78570.29 78734.55 0.21 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_7_sql_batch RuntimeMs DirectRunner 326788.15 364568.64 11.56 [1]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_sql_batch RuntimeMs SparkRunner 7212.82 7068.95 -1.99 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_sql_batch RuntimeMs SparkStructuredStreamingRunner 5996.48
>> 5910.11 -1.44 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_7_sql_streaming RuntimeMs DataflowRunner 1003130.36 1027200.23
>> 2.4 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_7_sql_streaming RuntimeMs DirectRunner 1076402.96 1215625.41
>> 12.93 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_sql_streaming RuntimeMs FlinkRunner 3486.11 3459.86 -0.75 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_streaming RuntimeMs DataflowRunner 120673.19 122350.28 1.39 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_7_streaming RuntimeMs DirectRunner 850907.18 848503.82 -0.28 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_7_streaming RuntimeMs FlinkRunner 13410.82 13769.05 2.67 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_8_batch RuntimeMs DataflowRunner 21572.36 21636.01 0.3 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_8_batch RuntimeMs DirectRunner 2258.14 2383.27 5.54 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_8_batch RuntimeMs FlinkRunner 473.39 478.09 0.99 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_8_batch RuntimeMs SparkRunner 1451.93 1434.59 -1.19 [1]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_8_batch RuntimeMs SparkStructuredStreamingRunner 1185.75 1128.57
>> -4.82 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_8_streaming RuntimeMs DataflowRunner 59645.41 61147.95 2.52 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_8_streaming RuntimeMs DirectRunner 6981.75 7187.59 2.95 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_8_streaming RuntimeMs FlinkRunner 479.46 478.59 -0.18 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_9_batch RuntimeMs DataflowRunner 29188.29 30279.18 3.74 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_9_batch RuntimeMs DirectRunner 1454.29 1440.0 -0.98 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_9_batch RuntimeMs FlinkRunner 290.18 325.45 12.16 [1]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_9_batch RuntimeMs SparkRunner 541.73 544.66 0.54 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_9_batch RuntimeMs SparkStructuredStreamingRunner 708.32 716.98
>> 1.22 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
>> <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_9_streaming RuntimeMs DataflowRunner 31862.82 31803.18 -0.19 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_9_streaming RuntimeMs DirectRunner 1758.86 1774.09 0.87 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_9_streaming RuntimeMs FlinkRunner 430.46 487.23 13.19 [1]
>> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
>> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>>
>>
>>
>
Re: Beam Metrics Report (2022-07-12)
Posted by Alexey Romanenko <ar...@gmail.com>.
Thanks Danny for your thoughts, I totally agree with you on a short and a long term.
IINM, this report is created and sent via “report_generator.py" script that reads metrics from InfluxDB.
Since we moved to Grafana for dashboards, I’m wondering if we can achieve the similar things, that you proposed, with Grafana alerts mechanism. Though, I guess it won’t be evident to find a commit that potentially caused a performance degradation.
—
Alexey
> On 14 Jul 2022, at 16:43, Danny McCormick via dev <de...@beam.apache.org> wrote:
>
> I like the idea of doing something like that, but using the median does mean we're likely going to be slower to detect regressions - for example, if we have an uptick on the 4th-7th day of the week, the median will not normally be affected. I would prefer something like excluding the top and bottom 5% of data points and then taking the mean. With that said, I think median is better than the current simple mean approach, so if that's an easier step I support it.
>
> In general, it would be awesome if we could make these more actionable (or replace them with something more actionable). My north star would be, for each performance test run:
>
> 1) Detect if performance has improved or degraded by more than ~1.5 standard deviations from the median of the last 10 week.
> 2) If so, do several sample runs at the old commit and new commit to see if the regression is real.
>
> That would give us a commit range where a regression was introduced, which we could alert on. If we wanted, we could even take it a step further and:
>
> 3) Automatically do a git bisect to find the commit where performance degraded.
>
> That's longer term and should not block us from making shorter term improvements like what you're describing.
>
> Thanks,
> Danny
>
> On Thu, Jul 14, 2022 at 7:16 AM Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
> Taking a look on red possible regressions number, I notice that it can be caused by some outliers that may be happened because of, for example, resources issues on Jenkins.
>
> <Screenshot 2022-07-14 at 13.08.14.png>
>
> (nexmark_11_batch, DataflowRunner)
>
>
> Should we use a median value instead of a mean one to make it more representative? Thoughts?
>
> —
> Alexey
>
>> On 12 Jul 2022, at 13:55, Apache Jenkins Server <jenkins@builds.apache.org <ma...@builds.apache.org>> wrote:
>>
>> Beam Metrics Report
>>
>> Color legend:
>> >= 20 >= 10 <= -10 <= -20
>> Possible regression
>>
>> Measurement Metric Runner Mean previous week Mean last week Diff % Dashboard
>> java_batch_cogbk_3 dataflow_v2_java17_runtime_sec - 13.58 16.39 20.7 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_4 dataflow_v2_java11_runtime_sec - 14.25 15.78 10.76 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_gbk_3 dataflow_v2_java11_runtime_sec - 14.44 17.31 19.88 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_v2_java17_runtime_sec - 35.86 39.61 10.45 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 sparkstructuredstreaming_runtime_sec - 84.12 92.67 10.17 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_pardo_1 dataflow_v2_java11_runtime_sec - 13.25 16.08 21.34 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_1 dataflow_v2_java17_runtime_sec - 15.09 16.7 10.66 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_2 dataflow_v2_java11_runtime_sec - 63.83 71.69 12.31 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_v2_java11_runtime_sec - 9.09 15.75 73.32 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_v2_java17_runtime_sec - 12.12 14.5 19.67 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_gbk_3 dataflow_v2_java11_runtime_sec - 53.16 59.27 11.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_pardo_3 dataflow_v2_java11_runtime_sec - 21.3 23.56 10.62 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> kafkaioit_results_runner_v2 read_time - 1.87 2.36 26.33 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> python_batch_cogbk_2 python_dataflow_batch_cogbk_2_runtime - 287.43 317.2 10.36 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_gbk_3 python_dataflow_batch_gbk_3_runtime - 27.43 32.6 18.85 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_streaming_pardo_4 python_dataflow_streaming_pardo_4_runtime - 538.57 602.0 11.78 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> nexmark_11_batch RuntimeMs DataflowRunner 39589.51 61325.57 54.9 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_16_batch RuntimeMs FlinkRunner 214.46 256.23 19.47 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_7_sql_batch RuntimeMs DirectRunner 326788.15 364568.64 11.56 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_sql_streaming RuntimeMs DirectRunner 1076402.96 1215625.41 12.93 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_9_batch RuntimeMs FlinkRunner 290.18 325.45 12.16 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_9_streaming RuntimeMs FlinkRunner 430.46 487.23 13.19 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> Possible improvement
>>
>> Measurement Metric Runner Mean previous week Mean last week Diff % Dashboard
>> go_batch_gbk_7 dataflow_runtime - 125.77 111.91 -11.02 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_pardo_2 dataflow_runtime - 420.66 360.73 -14.25 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_cogbk_2 dataflow_v2_java11_runtime_sec - 50.65 44.37 -12.41 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_gbk_3 dataflow_runtime_sec - 11.22 10.07 -10.3 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_6 dataflow_v2_java17_runtime_sec - 31.33 26.3 -16.06 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_runtime_sec - 49.25 44.03 -10.62 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_v2_java11_runtime_sec - 42.23 34.58 -18.11 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_pardo_1 sparkstructuredstreaming_runtime_sec - 56.17 49.55 -11.8 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_runtime_sec - 13.8 12.29 -10.98 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_cogbk_1 dataflow_v2_java11_runtime_sec - 3095.03 2628.35 -15.08 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_gbk_1 dataflow_v2_java17_runtime_sec - 1003.04 897.89 -10.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_2 dataflow_v2_java11_runtime_sec - 4656.02 2798.25 -39.9 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_3 dataflow_v2_java17_runtime_sec - 69.2 53.75 -22.32 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_5 dataflow_v2_java17_runtime_sec - 677.63 555.04 -18.09 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_pardo_1 dataflow_v2_java11_runtime_sec - 28.86 23.28 -19.35 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_4 dataflow_v2_java17_runtime_sec - 37.98 33.86 -10.85 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> kafkaioit_results read_time - 777.55 0.0 -100.0 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results write_time - 212.65 14.37 -93.24 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results_sdf_wrapper write_time - 6.25 3.97 -36.43 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> python_batch_cogbk_1 python_dataflow_batch_cogbk_1_runtime - 525.29 453.8 -13.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_cogbk_3 python_dataflow_batch_cogbk_3_runtime - 282.71 253.8 -10.23 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_combine_4 python_dataflow_batch_combine_4_runtime - 61.86 53.0 -14.32 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> python_batch_sideinput_2 python_dataflow_batch_sideinput_2_runtime - 50.14 45.0 -10.26 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_6 python_dataflow_batch_sideinput_6_runtime - 45.29 38.0 -16.09 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_bqio_write_10GB_results bqio_write_10GB_results_runtime - 247.86 210.2 -15.19 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
>> python_streaming_gbk_3 python_dataflow_streaming_gbk_3_runtime - 90.86 78.5 -13.6 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> nexmark_11_batch RuntimeMs SparkStructuredStreamingRunner 1135.95 1015.14 -10.64 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_sql_streaming RuntimeMs DataflowRunner 42997.25 34577.23 -19.58 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_streaming RuntimeMs DataflowRunner 28812.61 24063.0 -16.48 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_15_batch RuntimeMs FlinkRunner 646.39 579.05 -10.42 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_2_batch RuntimeMs FlinkRunner 219.18 192.68 -12.09 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> All metrics
>>
>> Measurement Metric Runner Mean previous week Mean last week Diff % Dashboard
>> avroioit_hdfs_results read_time - 88.5 90.3 2.04 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> avroioit_hdfs_results write_time - 117.12 121.15 3.43 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> avroioit_results read_time - 90.57 93.45 3.18 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> avroioit_results write_time - 116.22 119.98 3.23 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> bqio_10GB_results_java_batch_avro avro_write_time - 35.46 34.18 -3.61 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> bqio_10GB_results_java_batch_avro read_time - 41.22 41.64 1.02 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> bqio_10GB_results_java_batch_json read_time - 42.36 38.94 -8.07 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> bqio_10GB_results_java_batch_json write_time - 51.59 50.42 -2.26 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> compressed_textioit_hdfs_results read_time - 158.32 160.91 1.64 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> compressed_textioit_hdfs_results write_time - 207.37 207.08 -0.14 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> compressed_textioit_results read_time - 158.41 161.48 1.94 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> compressed_textioit_results write_time - 204.95 204.34 -0.3 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> go_batch_cogbk_1 dataflow_runtime - 256.19 251.49 -1.84 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> go_batch_cogbk_2 dataflow_runtime - 148.89 143.6 -3.55 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> go_batch_cogbk_3 dataflow_runtime - 115.23 112.94 -1.99 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> go_batch_cogbk_4 dataflow_runtime - 114.24 112.66 -1.38 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> go_batch_combine_1 dataflow_runtime - 388.21 377.26 -2.82 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> go_batch_combine_4 dataflow_runtime - 77.88 76.26 -2.09 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> go_batch_combine_5 dataflow_runtime - 72.59 71.3 -1.78 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> go_batch_gbk_1 dataflow_runtime - 371.28 369.86 -0.38 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_2 dataflow_runtime - 100.97 98.34 -2.6 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_3 dataflow_runtime - 31.82 30.17 -5.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_4 dataflow_runtime - 55.12 54.68 -0.78 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_5 dataflow_runtime - 45.18 45.82 1.42 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_6 dataflow_runtime - 102.59 103.03 0.44 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_gbk_7 dataflow_runtime - 125.77 111.91 -11.02 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> go_batch_pardo_1 dataflow_runtime - 75.89 78.22 3.07 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> go_batch_pardo_2 dataflow_runtime - 420.66 360.73 -14.25 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> go_batch_pardo_3 dataflow_runtime - 63.86 65.18 2.07 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> go_batch_pardo_4 dataflow_runtime - 72.36 70.43 -2.67 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> go_batch_sideinput_3 dataflow_runtime - 2.35 2.44 3.86 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> go_batch_sideinput_4 dataflow_runtime - 208.13 206.1 -0.98 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> hadoopformatioit_results read_time - 6.33 6.34 0.13 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> hadoopformatioit_results write_time - 7.79 7.51 -3.63 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> java_batch_cogbk_1 dataflow_runtime_sec - 41.75 42.28 1.27 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_1 dataflow_v2_java11_runtime_sec - 46.39 43.44 -6.36 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_1 dataflow_v2_java17_runtime_sec - 49.57 45.83 -7.54 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_1 sparkstructuredstreaming_runtime_sec - 118.05 124.41 5.39 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_2 dataflow_runtime_sec - 39.63 42.27 6.67 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_2 dataflow_v2_java11_runtime_sec - 50.65 44.37 -12.41 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_2 dataflow_v2_java17_runtime_sec - 44.58 44.66 0.18 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_2 sparkstructuredstreaming_runtime_sec - 116.79 122.12 4.56 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_3 dataflow_runtime_sec - 15.17 15.82 4.32 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_3 dataflow_v2_java11_runtime_sec - 14.41 13.74 -4.68 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_3 dataflow_v2_java17_runtime_sec - 13.58 16.39 20.7 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_3 sparkstructuredstreaming_runtime_sec - 19.96 20.61 3.28 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_4 dataflow_runtime_sec - 13.12 13.65 4.05 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_4 dataflow_v2_java11_runtime_sec - 14.25 15.78 10.76 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_4 dataflow_v2_java17_runtime_sec - 13.23 13.71 3.59 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_cogbk_4 sparkstructuredstreaming_runtime_sec - 20.37 20.12 -1.19 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_batch_combine_1 dataflow_runtime_sec - 64.0 62.62 -2.16 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_combine_1 sparkstructuredstreaming_runtime_sec - 554.59 557.39 0.5 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_combine_4 dataflow_runtime_sec - 27.99 27.76 -0.8 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_combine_4 sparkstructuredstreaming_runtime_sec - 124.67 126.83 1.73 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_combine_5 dataflow_runtime_sec - 27.27 24.84 -8.93 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_combine_5 sparkstructuredstreaming_runtime_sec - 119.92 120.06 0.12 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_batch_gbk_1 dataflow_runtime_sec - 125.42 115.83 -7.64 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_1 dataflow_v2_java11_runtime_sec - 96.68 95.35 -1.38 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_1 dataflow_v2_java17_runtime_sec - 86.51 90.35 4.44 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_1 sparkstructuredstreaming_runtime_sec - 502.91 503.81 0.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_2 dataflow_runtime_sec - 33.68 32.33 -4.0 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_2 dataflow_v2_java11_runtime_sec - 42.27 40.93 -3.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_2 dataflow_v2_java17_runtime_sec - 39.31 40.94 4.12 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_2 sparkstructuredstreaming_runtime_sec - 97.8 107.11 9.52 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_3 dataflow_runtime_sec - 11.22 10.07 -10.3 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_3 dataflow_v2_java11_runtime_sec - 14.44 17.31 19.88 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_3 dataflow_v2_java17_runtime_sec - 15.28 15.39 0.75 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_3 sparkstructuredstreaming_runtime_sec - 8.2 8.74 6.56 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_4 dataflow_runtime_sec - 18.85 19.23 2.05 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_4 dataflow_v2_java11_runtime_sec - 22.58 22.18 -1.78 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_4 dataflow_v2_java17_runtime_sec - 23.02 24.62 6.92 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_4 sparkstructuredstreaming_runtime_sec - 97.45 101.76 4.43 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_5 dataflow_runtime_sec - 18.5 17.83 -3.64 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_5 dataflow_v2_java11_runtime_sec - 17.53 17.45 -0.49 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_5 dataflow_v2_java17_runtime_sec - 19.43 20.54 5.68 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_5 sparkstructuredstreaming_runtime_sec - 100.52 100.52 -0.0 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_6 dataflow_runtime_sec - 37.2 34.88 -6.26 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_6 dataflow_v2_java11_runtime_sec - 26.4 26.36 -0.12 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_6 dataflow_v2_java17_runtime_sec - 31.33 26.3 -16.06 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_6 sparkstructuredstreaming_runtime_sec - 58.34 60.86 4.32 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_runtime_sec - 49.25 44.03 -10.62 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_v2_java11_runtime_sec - 42.23 34.58 -18.11 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 dataflow_v2_java17_runtime_sec - 35.86 39.61 10.45 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_gbk_7 sparkstructuredstreaming_runtime_sec - 84.12 92.67 10.17 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_batch_pardo_1 dataflow_runtime_sec - 20.04 18.14 -9.49 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_1 dataflow_v2_java11_runtime_sec - 13.25 16.08 21.34 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_1 dataflow_v2_java17_runtime_sec - 15.09 16.7 10.66 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_1 sparkstructuredstreaming_runtime_sec - 56.17 49.55 -11.8 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_2 dataflow_runtime_sec - 186.08 202.59 8.87 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_2 dataflow_v2_java11_runtime_sec - 63.83 71.69 12.31 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_2 dataflow_v2_java17_runtime_sec - 66.38 67.13 1.13 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_runtime_sec - 13.8 12.29 -10.98 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_v2_java11_runtime_sec - 9.09 15.75 73.32 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_3 dataflow_v2_java17_runtime_sec - 12.12 14.5 19.67 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_4 dataflow_runtime_sec - 24.8 22.52 -9.18 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_4 dataflow_v2_java11_runtime_sec - 26.58 24.46 -7.98 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_batch_pardo_4 dataflow_v2_java17_runtime_sec - 25.41 24.93 -1.9 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_cogbk_1 dataflow_runtime_sec - 927.84 858.74 -7.45 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_1 dataflow_v2_java11_runtime_sec - 3095.03 2628.35 -15.08 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_1 dataflow_v2_java17_runtime_sec - 2589.29 2595.64 0.25 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_2 dataflow_runtime_sec - 907.41 837.58 -7.7 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_2 dataflow_v2_java11_runtime_sec - 3256.56 2969.53 -8.81 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_2 dataflow_v2_java17_runtime_sec - 2938.01 3090.66 5.2 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_3 dataflow_runtime_sec - 118.04 122.42 3.71 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_3 dataflow_v2_java11_runtime_sec - 322.79 326.12 1.03 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_3 dataflow_v2_java17_runtime_sec - 319.04 325.44 2.01 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_4 dataflow_runtime_sec - 125.95 122.06 -3.09 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_4 dataflow_v2_java11_runtime_sec - 320.92 326.47 1.73 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_cogbk_4 dataflow_v2_java17_runtime_sec - 331.9 322.66 -2.78 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> java_streaming_combine_1 dataflow_runtime_sec - 509.03 511.79 0.54 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_streaming_combine_4 dataflow_runtime_sec - 318.94 322.22 1.03 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_streaming_combine_5 dataflow_runtime_sec - 307.67 309.5 0.6 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> java_streaming_gbk_1 dataflow_runtime_sec - 670.68 646.17 -3.65 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_1 dataflow_v2_java11_runtime_sec - 875.0 920.0 5.14 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_1 dataflow_v2_java17_runtime_sec - 1003.04 897.89 -10.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_2 dataflow_runtime_sec - 803.33 798.42 -0.61 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_2 dataflow_v2_java11_runtime_sec - 4656.02 2798.25 -39.9 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_2 dataflow_v2_java17_runtime_sec - 3158.86 3166.16 0.23 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_3 dataflow_runtime_sec - 23.62 24.92 5.49 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_3 dataflow_v2_java11_runtime_sec - 53.16 59.27 11.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_3 dataflow_v2_java17_runtime_sec - 69.2 53.75 -22.32 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_4 dataflow_runtime_sec - 289.5 286.49 -1.04 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_4 dataflow_v2_java11_runtime_sec - 641.36 649.91 1.33 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_4 dataflow_v2_java17_runtime_sec - 665.68 685.61 2.99 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_5 dataflow_runtime_sec - 293.31 280.95 -4.21 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_5 dataflow_v2_java11_runtime_sec - 602.18 632.83 5.09 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_5 dataflow_v2_java17_runtime_sec - 677.63 555.04 -18.09 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_6 dataflow_runtime_sec - 73.86 76.09 3.02 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_6 dataflow_v2_java11_runtime_sec - 160.18 145.57 -9.12 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_6 dataflow_v2_java17_runtime_sec - 194.24 178.72 -7.99 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_7 dataflow_runtime_sec - 137.68 137.62 -0.04 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_7 dataflow_v2_java11_runtime_sec - 207.62 226.19 8.94 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_gbk_7 dataflow_v2_java17_runtime_sec - 223.14 202.22 -9.37 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> java_streaming_pardo_1 dataflow_runtime_sec - 18.79 18.93 0.75 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_1 dataflow_v2_java11_runtime_sec - 28.86 23.28 -19.35 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_1 dataflow_v2_java17_runtime_sec - 25.27 24.23 -4.08 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_2 dataflow_runtime_sec - 223.51 226.23 1.22 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_2 dataflow_v2_java11_runtime_sec - 114.31 111.55 -2.41 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_2 dataflow_v2_java17_runtime_sec - 105.51 97.51 -7.58 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_3 dataflow_runtime_sec - 12.55 12.71 1.25 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_3 dataflow_v2_java11_runtime_sec - 21.3 23.56 10.62 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_3 dataflow_v2_java17_runtime_sec - 24.03 24.17 0.61 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_4 dataflow_runtime_sec - 32.98 33.47 1.49 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_4 dataflow_v2_java11_runtime_sec - 36.78 35.46 -3.61 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> java_streaming_pardo_4 dataflow_v2_java17_runtime_sec - 37.98 33.86 -10.85 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> jdbcioit_results read_time - 24.89 24.82 -0.28 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> jdbcioit_results write_time - 38.16 36.52 -4.3 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results read_time - 777.55 0.0 -100.0 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results write_time - 212.65 14.37 -93.24 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results_runner_v2 read_time - 1.87 2.36 26.33 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results_runner_v2 write_time - 5.89 5.91 0.4 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results_sdf_wrapper read_time - 3.04 3.26 7.08 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> kafkaioit_results_sdf_wrapper write_time - 6.25 3.97 -36.43 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> many_files_textioit_hdfs_results read_time - 17.29 17.48 1.13 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> many_files_textioit_hdfs_results write_time - 61.16 56.82 -7.09 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> many_files_textioit_results read_time - 24.9 24.23 -2.67 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> many_files_textioit_results write_time - 61.41 58.16 -5.29 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> mongodbioit_results read_time - 30.18 30.83 2.15 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> mongodbioit_results write_time - 59.02 55.45 -6.06 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> parquetioit_hdfs_results read_time - 88.2 88.97 0.88 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> parquetioit_hdfs_results write_time - 147.1 142.12 -3.39 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> parquetioit_results read_time - 87.88 89.9 2.29 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> parquetioit_results write_time - 145.06 139.23 -4.02 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> python_batch_cogbk_1 python_dataflow_batch_cogbk_1_runtime - 525.29 453.8 -13.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_cogbk_2 python_dataflow_batch_cogbk_2_runtime - 287.43 317.2 10.36 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_cogbk_3 python_dataflow_batch_cogbk_3_runtime - 282.71 253.8 -10.23 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_cogbk_4 python_dataflow_batch_cogbk_4_runtime - 291.71 278.4 -4.56 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_batch_combine_1 python_dataflow_batch_combine_1_runtime - 841.29 875.6 4.08 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> python_batch_combine_4 python_dataflow_batch_combine_4_runtime - 61.86 53.0 -14.32 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> python_batch_combine_5 python_dataflow_batch_combine_5_runtime - 46.0 44.4 -3.48 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
>> python_batch_gbk_1 python_dataflow_batch_gbk_1_runtime - 1877.29 1915.8 2.05 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_2 python_dataflow_batch_gbk_2_runtime - 373.14 367.8 -1.43 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_3 python_dataflow_batch_gbk_3_runtime - 27.43 32.6 18.85 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_4 python_dataflow_batch_gbk_4_runtime - 124.43 124.2 -0.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_5 python_dataflow_batch_gbk_5_runtime - 127.0 129.4 1.89 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_6 python_dataflow_batch_gbk_6_runtime - 344.86 352.4 2.19 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_gbk_7 python_dataflow_batch_gbk_7_runtime - 353.14 346.0 -2.02 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_batch_pardo_1 python_dataflow_batch_pardo_1_runtime - 180.14 176.8 -1.86 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_batch_pardo_2 python_dataflow_batch_pardo_2_runtime - 1635.57 1661.4 1.58 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_batch_pardo_3 python_dataflow_batch_pardo_3_runtime - 141.86 150.6 6.16 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_batch_pardo_4 python_dataflow_batch_pardo_4_runtime - 326.71 345.2 5.66 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_batch_sideinput_1 python_dataflow_batch_sideinput_1_runtime - 47.86 46.4 -3.04 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_2 python_dataflow_batch_sideinput_2_runtime - 50.14 45.0 -10.26 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_3 python_dataflow_batch_sideinput_3_runtime - 5.71 5.4 -5.5 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_4 python_dataflow_batch_sideinput_4_runtime - 471.29 436.4 -7.4 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_5 python_dataflow_batch_sideinput_5_runtime - 47.29 44.4 -6.1 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_6 python_dataflow_batch_sideinput_6_runtime - 45.29 38.0 -16.09 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_7 python_dataflow_batch_sideinput_7_runtime - 5566.29 5588.2 0.39 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_batch_sideinput_8 python_dataflow_batch_sideinput_8_runtime - 5592.14 5562.8 -0.52 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
>> python_bqio_read_10GB_results bqio_read_10GB_results_runtime - 118.71 111.6 -5.99 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
>> python_bqio_write_10GB_results bqio_write_10GB_results_runtime - 247.86 210.2 -15.19 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
>> python_direct_microbenchmarks fn_api_runner_microbenchmark_runtime_sec - 94.83 94.11 -0.76
>> python_direct_microbenchmarks teststream_microbenchmark_runtime_sec - 344.58 337.86 -1.95
>> python_spanner_write_2GB_results pyspanner_write_2GB_results_runtime - 1562.57 1599.2 2.34 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
>> python_spannerio_read_2GB_results pyspanner_read_2GB_results_runtime - 26.43 23.8 -9.95 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
>> python_streaming_cogbk_1 python_dataflow_streaming_cogbk_1_runtime - 628.29 659.8 5.02 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_streaming_cogbk_2 python_dataflow_streaming_cogbk_2_runtime - 425.0 422.4 -0.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_streaming_cogbk_3 python_dataflow_streaming_cogbk_3_runtime - 506.43 491.2 -3.01 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_streaming_cogbk_4 python_dataflow_streaming_cogbk_4_runtime - 424.71 427.0 0.54 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
>> python_streaming_gbk_3 python_dataflow_streaming_gbk_3_runtime - 90.86 78.5 -13.6 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_streaming_gbk_6 python_dataflow_streaming_gbk_6_runtime - 523.57 546.0 4.28 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_streaming_gbk_7 python_dataflow_streaming_gbk_7_runtime - 506.29 553.5 9.33 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
>> python_streaming_pardo_1 python_dataflow_streaming_pardo_1_runtime - 253.57 235.5 -7.13 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_streaming_pardo_2 python_dataflow_streaming_pardo_2_runtime - 2941.57 3096.5 5.27 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_streaming_pardo_3 python_dataflow_streaming_pardo_3_runtime - 204.0 200.0 -1.96 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> python_streaming_pardo_4 python_dataflow_streaming_pardo_4_runtime - 538.57 602.0 11.78 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
>> textioit_hdfs_results read_time - 19.88 20.73 4.27 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> textioit_hdfs_results write_time - 32.34 31.18 -3.59 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> textioit_results read_time - 20.52 21.58 5.16 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> textioit_results write_time - 32.12 32.81 2.14 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> tfrecordioit_results read_time - 15.43 16.26 5.35 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> tfrecordioit_results write_time - 20.79 21.14 1.68 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> wordcount_py37_results runtime - 811.08 824.06 1.6 [1] <http://metrics.beam.apache.org/d/1cnwVDkGk>
>> xmlioit_hdfs_results read_time - 21.4 22.77 6.44 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> xmlioit_hdfs_results write_time - 24.67 25.12 1.79 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> xmlioit_results read_time - 23.48 23.36 -0.53 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> xmlioit_results write_time - 24.44 26.24 7.36 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
>> nexmark_0_batch RuntimeMs DataflowRunner 27146.02 27212.08 0.24 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_batch RuntimeMs DirectRunner 4782.36 4783.0 0.01 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_batch RuntimeMs FlinkRunner 1155.07 1111.86 -3.74 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_0_batch RuntimeMs SparkRunner 1253.64 1255.18 0.12 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_batch RuntimeMs SparkStructuredStreamingRunner 1546.91 1554.18 0.47 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_0_sql_batch RuntimeMs DataflowRunner 35274.21 35610.5 0.95 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_0_sql_batch RuntimeMs DirectRunner 5570.26 5655.59 1.53 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_sql_batch RuntimeMs SparkRunner 3205.2 3141.48 -1.99 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_0_sql_batch RuntimeMs SparkStructuredStreamingRunner 2939.46 2926.11 -0.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_0_sql_streaming RuntimeMs DataflowRunner 24026.89 23121.55 -3.77 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_sql_streaming RuntimeMs DirectRunner 15501.0 15555.91 0.35 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_sql_streaming RuntimeMs FlinkRunner 1637.46 1628.64 -0.54 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_0_streaming RuntimeMs DataflowRunner 39291.92 37816.14 -3.76 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_streaming RuntimeMs DirectRunner 11811.79 11938.45 1.07 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_streaming RuntimeMs FlinkRunner 1432.14 1422.36 -0.68 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_0_zetasql_batch RuntimeMs DataflowRunner 114915.96 112238.45 -2.33 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_0_zetasql_batch RuntimeMs DirectRunner 6757.07 6834.27 1.14 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_0_zetasql_streaming RuntimeMs DataflowRunner 67751.04 66627.91 -1.66 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_zetasql_streaming RuntimeMs DirectRunner 17105.75 16996.45 -0.64 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_0_zetasql_streaming RuntimeMs FlinkRunner 3248.54 3261.41 0.4 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_10_batch RuntimeMs DataflowRunner 48690.3 49306.87 1.27 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_10_batch RuntimeMs DirectRunner 10463.32 10465.09 0.02 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_10_batch RuntimeMs FlinkRunner 777.21 714.64 -8.05 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_10_batch RuntimeMs SparkRunner 726.5 729.61 0.43 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_10_batch RuntimeMs SparkStructuredStreamingRunner 1126.52 1211.0 7.5 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_10_streaming RuntimeMs DataflowRunner 105776.61 102899.66 -2.72 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_10_streaming RuntimeMs DirectRunner 43926.21 44091.14 0.38 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_10_streaming RuntimeMs FlinkRunner 1535.29 1488.14 -3.07 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_11_batch RuntimeMs DataflowRunner 39589.51 61325.57 54.9 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_11_batch RuntimeMs DirectRunner 3757.79 3770.55 0.34 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_11_batch RuntimeMs FlinkRunner 297.43 313.64 5.45 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_11_batch RuntimeMs SparkRunner 1402.09 1399.52 -0.18 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_11_batch RuntimeMs SparkStructuredStreamingRunner 1135.95 1015.14 -10.64 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_11_streaming RuntimeMs DataflowRunner 182507.05 181193.81 -0.72 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_11_streaming RuntimeMs DirectRunner 10696.0 10739.0 0.4 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_11_streaming RuntimeMs FlinkRunner 1351.79 1319.05 -2.42 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_12_batch RuntimeMs DataflowRunner 19630.69 19666.75 0.18 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_12_batch RuntimeMs DirectRunner 2597.07 2626.36 1.13 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_12_batch RuntimeMs FlinkRunner 180.29 173.05 -4.02 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_12_batch RuntimeMs SparkRunner 924.52 929.25 0.51 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_12_batch RuntimeMs SparkStructuredStreamingRunner 827.98 899.98 8.7 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_12_streaming RuntimeMs DataflowRunner 96217.51 94780.91 -1.49 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_12_streaming RuntimeMs DirectRunner 9335.68 9409.5 0.79 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_12_streaming RuntimeMs FlinkRunner 621.5 620.73 -0.12 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_batch RuntimeMs DataflowRunner 33870.18 33840.36 -0.09 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_batch RuntimeMs DirectRunner 3225.36 3231.86 0.2 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_13_batch RuntimeMs FlinkRunner 392.68 404.95 3.13 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_13_batch RuntimeMs SparkRunner 355.27 357.02 0.49 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_batch RuntimeMs SparkStructuredStreamingRunner 547.32 556.57 1.69 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_13_sql_batch RuntimeMs DataflowRunner 62806.75 61308.68 -2.39 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_13_sql_batch RuntimeMs DirectRunner 9065.15 9052.5 -0.14 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_sql_batch RuntimeMs SparkRunner 3473.45 3430.11 -1.25 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_sql_batch RuntimeMs SparkStructuredStreamingRunner 3113.52 3060.59 -1.7 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_sql_streaming RuntimeMs DataflowRunner 42997.25 34577.23 -19.58 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_sql_streaming RuntimeMs DirectRunner 18918.29 18631.09 -1.52 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_sql_streaming RuntimeMs FlinkRunner 28617.96 29781.77 4.07 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_streaming RuntimeMs DataflowRunner 28812.61 24063.0 -16.48 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_streaming RuntimeMs DirectRunner 9996.14 9949.18 -0.47 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_streaming RuntimeMs FlinkRunner 27765.36 28396.95 2.27 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_zetasql_batch RuntimeMs DataflowRunner 262683.96 259338.23 -1.27 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_13_zetasql_batch RuntimeMs DirectRunner 10046.04 10059.45 0.13 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_13_zetasql_streaming RuntimeMs DataflowRunner 116689.18 109084.86 -6.52 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_zetasql_streaming RuntimeMs DirectRunner 21036.07 21342.05 1.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_13_zetasql_streaming RuntimeMs FlinkRunner 34089.0 34659.64 1.67 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_14_batch RuntimeMs DataflowRunner 62350.36 61308.36 -1.67 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_14_batch RuntimeMs DirectRunner 4153.39 4150.05 -0.08 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_14_batch RuntimeMs FlinkRunner 666.86 650.5 -2.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_14_batch RuntimeMs SparkRunner 895.64 945.5 5.57 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_14_batch RuntimeMs SparkStructuredStreamingRunner 933.43 941.27 0.84 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_14_streaming RuntimeMs DataflowRunner 121083.79 126024.55 4.08 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_14_streaming RuntimeMs DirectRunner 12338.79 12202.36 -1.11 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_14_streaming RuntimeMs FlinkRunner 1783.46 1762.27 -1.19 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_15_batch RuntimeMs DataflowRunner 45464.97 45823.83 0.79 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_15_batch RuntimeMs DirectRunner 15421.04 15413.5 -0.05 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_15_batch RuntimeMs FlinkRunner 646.39 579.05 -10.42 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_15_batch RuntimeMs SparkRunner 1643.96 1675.16 1.9 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_15_batch RuntimeMs SparkStructuredStreamingRunner 1451.93 1442.11 -0.68 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_16_batch RuntimeMs DataflowRunner 51963.45 51419.82 -1.05 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_16_batch RuntimeMs DirectRunner 85652.86 84922.68 -0.85 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_16_batch RuntimeMs FlinkRunner 214.46 256.23 19.47 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_16_batch RuntimeMs SparkRunner 1484.05 1386.3 -6.59 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_16_batch RuntimeMs SparkStructuredStreamingRunner 1249.75 1199.98 -3.98 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_16_streaming RuntimeMs DataflowRunner 89716.22 95450.93 6.39 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_16_streaming RuntimeMs DirectRunner 124442.11 129482.0 4.05 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_16_streaming RuntimeMs FlinkRunner 423.43 431.27 1.85 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_batch RuntimeMs DataflowRunner 21877.32 20846.38 -4.71 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_batch RuntimeMs DirectRunner 3502.18 3394.41 -3.08 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_1_batch RuntimeMs FlinkRunner 315.89 307.91 -2.53 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_1_batch RuntimeMs SparkRunner 440.91 442.09 0.27 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_1_batch RuntimeMs SparkStructuredStreamingRunner 591.54 592.73 0.2 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_1_sql_batch RuntimeMs DataflowRunner 32344.21 32125.86 -0.68 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_1_sql_batch RuntimeMs DirectRunner 3546.3 3726.64 5.09 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_sql_batch RuntimeMs SparkRunner 2430.38 2390.05 -1.66 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_1_sql_batch RuntimeMs SparkStructuredStreamingRunner 1572.41 1666.39 5.98 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_1_sql_streaming RuntimeMs DataflowRunner 20870.43 21097.45 1.09 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_1_sql_streaming RuntimeMs DirectRunner 11466.04 11491.86 0.23 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_sql_streaming RuntimeMs FlinkRunner 667.54 637.77 -4.46 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_streaming RuntimeMs DataflowRunner 28937.91 28601.91 -1.16 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_streaming RuntimeMs DirectRunner 8206.14 8198.05 -0.1 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_1_streaming RuntimeMs FlinkRunner 648.86 667.5 2.87 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_batch RuntimeMs DataflowRunner 15263.96 15124.28 -0.92 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_2_batch RuntimeMs DirectRunner 1530.89 1605.64 4.88 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_2_batch RuntimeMs FlinkRunner 219.18 192.68 -12.09 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_batch RuntimeMs SparkRunner 311.59 310.82 -0.25 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_batch RuntimeMs SparkStructuredStreamingRunner 458.84 460.77 0.42 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_2_sql_batch RuntimeMs DataflowRunner 25789.39 25085.68 -2.73 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_2_sql_batch RuntimeMs DirectRunner 1543.37 1555.64 0.79 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_2_sql_batch RuntimeMs SparkRunner 744.41 744.02 -0.05 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_sql_batch RuntimeMs SparkStructuredStreamingRunner 1005.04 972.02 -3.28 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_2_sql_streaming RuntimeMs DataflowRunner 13841.29 13565.55 -1.99 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_sql_streaming RuntimeMs DirectRunner 6789.04 6801.5 0.18 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_sql_streaming RuntimeMs FlinkRunner 424.61 410.73 -3.27 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_streaming RuntimeMs DataflowRunner 19257.58 18899.86 -1.86 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_streaming RuntimeMs DirectRunner 5142.04 5144.05 0.04 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_streaming RuntimeMs FlinkRunner 270.54 275.0 1.65 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_zetasql_batch RuntimeMs DataflowRunner 71538.04 73956.5 3.38 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_zetasql_batch RuntimeMs DirectRunner 2908.36 2883.55 -0.85 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_zetasql_streaming RuntimeMs DataflowRunner 39600.89 40732.09 2.86 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_2_zetasql_streaming RuntimeMs DirectRunner 8218.75 8148.45 -0.86 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_2_zetasql_streaming RuntimeMs FlinkRunner 1202.64 1165.27 -3.11 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_batch RuntimeMs DataflowRunner 26748.14 27266.41 1.94 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_batch RuntimeMs DirectRunner 3224.14 3243.5 0.6 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_batch RuntimeMs FlinkRunner 1105.75 1059.41 -4.19 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_3_batch RuntimeMs SparkRunner 2086.2 2081.61 -0.22 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_sql_batch RuntimeMs DataflowRunner 29239.46 29789.05 1.88 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_3_sql_batch RuntimeMs DirectRunner 3365.15 3329.5 -1.06 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_3_sql_batch RuntimeMs SparkRunner 3503.88 3480.3 -0.67 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_3_sql_batch RuntimeMs SparkStructuredStreamingRunner 3032.75 2996.09 -1.21 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_3_sql_streaming RuntimeMs DataflowRunner 33163.79 33241.18 0.23 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_sql_streaming RuntimeMs DirectRunner 8460.71 8474.64 0.16 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_sql_streaming RuntimeMs FlinkRunner 970.11 960.36 -1.0 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_streaming RuntimeMs DataflowRunner 43269.14 43554.49 0.66 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_streaming RuntimeMs DirectRunner 7623.46 7534.73 -1.16 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_streaming RuntimeMs FlinkRunner 469.46 456.77 -2.7 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_zetasql_batch RuntimeMs DataflowRunner 54330.46 54539.23 0.38 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_3_zetasql_batch RuntimeMs DirectRunner 3749.36 3745.59 -0.1 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_zetasql_streaming RuntimeMs DataflowRunner 46361.32 46243.59 -0.25 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_3_zetasql_streaming RuntimeMs DirectRunner 8915.93 8861.45 -0.61 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_3_zetasql_streaming RuntimeMs FlinkRunner 1191.43 1197.64 0.52 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_4_batch RuntimeMs DataflowRunner 33212.25 33292.5 0.24 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_4_batch RuntimeMs DirectRunner 2416.46 2409.05 -0.31 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_4_batch RuntimeMs FlinkRunner 861.68 851.14 -1.22 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_4_batch RuntimeMs SparkRunner 1001.52 998.98 -0.25 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_4_batch RuntimeMs SparkStructuredStreamingRunner 2243.59 2233.52 -0.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_4_streaming RuntimeMs DataflowRunner 49175.82 49133.95 -0.09 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_4_streaming RuntimeMs DirectRunner 2827.5 2862.05 1.22 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_4_streaming RuntimeMs FlinkRunner 975.32 1021.45 4.73 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_5_batch RuntimeMs DataflowRunner 35151.96 34758.3 -1.12 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_5_batch RuntimeMs DirectRunner 10582.36 10621.95 0.37 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_5_batch RuntimeMs FlinkRunner 775.96 769.41 -0.84 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_5_batch RuntimeMs SparkRunner 1515.27 1535.14 1.31 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_5_batch RuntimeMs SparkStructuredStreamingRunner 1719.73 1732.2 0.73 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_5_streaming RuntimeMs DataflowRunner 214447.03 209021.26 -2.53 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_5_streaming RuntimeMs DirectRunner 20728.25 21227.32 2.41 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_5_streaming RuntimeMs FlinkRunner 1609.86 1637.59 1.72 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_6_batch RuntimeMs DataflowRunner 31298.14 31001.5 -0.95 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_6_batch RuntimeMs DirectRunner 1886.82 1851.91 -1.85 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_6_batch RuntimeMs FlinkRunner 536.79 519.91 -3.14 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_6_batch RuntimeMs SparkRunner 710.09 706.14 -0.56 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_6_batch RuntimeMs SparkStructuredStreamingRunner 907.57 948.52 4.51 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_6_streaming RuntimeMs DataflowRunner 38287.43 41809.0 9.2 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_6_streaming RuntimeMs DirectRunner 2135.57 2098.95 -1.71 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_6_streaming RuntimeMs FlinkRunner 601.57 592.18 -1.56 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_batch RuntimeMs DataflowRunner 54975.39 56478.73 2.73 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_7_batch RuntimeMs DirectRunner 18420.82 18547.23 0.69 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_7_batch RuntimeMs FlinkRunner 665.07 677.59 1.88 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_7_batch RuntimeMs SparkRunner 1440.04 1426.75 -0.92 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_7_batch RuntimeMs SparkStructuredStreamingRunner 2214.61 2196.27 -0.83 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_7_sql_batch RuntimeMs DataflowRunner 78570.29 78734.55 0.21 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_7_sql_batch RuntimeMs DirectRunner 326788.15 364568.64 11.56 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_sql_batch RuntimeMs SparkRunner 7212.82 7068.95 -1.99 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_sql_batch RuntimeMs SparkStructuredStreamingRunner 5996.48 5910.11 -1.44 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_7_sql_streaming RuntimeMs DataflowRunner 1003130.36 1027200.23 2.4 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_7_sql_streaming RuntimeMs DirectRunner 1076402.96 1215625.41 12.93 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_sql_streaming RuntimeMs FlinkRunner 3486.11 3459.86 -0.75 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_7_streaming RuntimeMs DataflowRunner 120673.19 122350.28 1.39 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_7_streaming RuntimeMs DirectRunner 850907.18 848503.82 -0.28 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_7_streaming RuntimeMs FlinkRunner 13410.82 13769.05 2.67 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_8_batch RuntimeMs DataflowRunner 21572.36 21636.01 0.3 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_8_batch RuntimeMs DirectRunner 2258.14 2383.27 5.54 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_8_batch RuntimeMs FlinkRunner 473.39 478.09 0.99 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_8_batch RuntimeMs SparkRunner 1451.93 1434.59 -1.19 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_8_batch RuntimeMs SparkStructuredStreamingRunner 1185.75 1128.57 -4.82 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_8_streaming RuntimeMs DataflowRunner 59645.41 61147.95 2.52 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_8_streaming RuntimeMs DirectRunner 6981.75 7187.59 2.95 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_8_streaming RuntimeMs FlinkRunner 479.46 478.59 -0.18 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
>> nexmark_9_batch RuntimeMs DataflowRunner 29188.29 30279.18 3.74 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_9_batch RuntimeMs DirectRunner 1454.29 1440.0 -0.98 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
>> nexmark_9_batch RuntimeMs FlinkRunner 290.18 325.45 12.16 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_9_batch RuntimeMs SparkRunner 541.73 544.66 0.54 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_9_batch RuntimeMs SparkStructuredStreamingRunner 708.32 716.98 1.22 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
>> nexmark_9_streaming RuntimeMs DataflowRunner 31862.82 31803.18 -0.19 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_9_streaming RuntimeMs DirectRunner 1758.86 1774.09 0.87 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
>> nexmark_9_streaming RuntimeMs FlinkRunner 430.46 487.23 13.19 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
Re: Beam Metrics Report (2022-07-12)
Posted by Danny McCormick via dev <de...@beam.apache.org>.
I like the idea of doing something like that, but using the median does
mean we're likely going to be slower to detect regressions - for example,
if we have an uptick on the 4th-7th day of the week, the median will not
normally be affected. I would prefer something like excluding the top and
bottom 5% of data points and then taking the mean. With that said, I think
median is better than the current simple mean approach, so if that's an
easier step I support it.
In general, it would be awesome if we could make these more actionable (or
replace them with something more actionable). My north star would be, for
each performance test run:
1) Detect if performance has improved or degraded by more than ~1.5
standard deviations from the median of the last 10 week.
2) If so, do several sample runs at the old commit and new commit to see if
the regression is real.
That would give us a commit range where a regression was introduced, which
we could alert on. If we wanted, we could even take it a step further and:
3) Automatically do a git bisect to find the commit where performance
degraded.
That's longer term and should not block us from making shorter term
improvements like what you're describing.
Thanks,
Danny
On Thu, Jul 14, 2022 at 7:16 AM Alexey Romanenko <ar...@gmail.com>
wrote:
> Taking a look on red possible regressions number, I notice that it can be
> caused by some outliers that may be happened because of, for example,
> resources issues on Jenkins.
>
>
>
> (nexmark_11_batch, DataflowRunner)
>
>
> Should we use a median value instead of a mean one to make it more
> representative? Thoughts?
>
> —
> Alexey
>
> On 12 Jul 2022, at 13:55, Apache Jenkins Server <je...@builds.apache.org>
> wrote:
>
> Beam Metrics Report Color legend:
> >= 20 >= 10 <= -10 <= -20 Possible regression
> *Measurement* *Metric* *Runner* *Mean previous week* *Mean last week* *Diff
> %* *Dashboard*
> java_batch_cogbk_3 dataflow_v2_java17_runtime_sec - 13.58 16.39 20.7 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_4 dataflow_v2_java11_runtime_sec - 14.25 15.78 10.76 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_gbk_3 dataflow_v2_java11_runtime_sec - 14.44 17.31 19.88 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_v2_java17_runtime_sec - 35.86 39.61 10.45 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 sparkstructuredstreaming_runtime_sec - 84.12 92.67 10.17
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_pardo_1 dataflow_v2_java11_runtime_sec - 13.25 16.08 21.34 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_1 dataflow_v2_java17_runtime_sec - 15.09 16.7 10.66 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_2 dataflow_v2_java11_runtime_sec - 63.83 71.69 12.31 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_v2_java11_runtime_sec - 9.09 15.75 73.32 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_v2_java17_runtime_sec - 12.12 14.5 19.67 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_gbk_3 dataflow_v2_java11_runtime_sec - 53.16 59.27 11.48
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_pardo_3 dataflow_v2_java11_runtime_sec - 21.3 23.56 10.62
> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> kafkaioit_results_runner_v2 read_time - 1.87 2.36 26.33 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> python_batch_cogbk_2 python_dataflow_batch_cogbk_2_runtime - 287.43 317.2
> 10.36 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_gbk_3 python_dataflow_batch_gbk_3_runtime - 27.43 32.6 18.85
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_streaming_pardo_4 python_dataflow_streaming_pardo_4_runtime -
> 538.57 602.0 11.78 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> nexmark_11_batch RuntimeMs DataflowRunner 39589.51 61325.57 54.9 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_16_batch RuntimeMs FlinkRunner 214.46 256.23 19.47 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_7_sql_batch RuntimeMs DirectRunner 326788.15 364568.64 11.56 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_sql_streaming RuntimeMs DirectRunner 1076402.96 1215625.41 12.93
> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_9_batch RuntimeMs FlinkRunner 290.18 325.45 12.16 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_9_streaming RuntimeMs FlinkRunner 430.46 487.23 13.19 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> Possible improvement
> *Measurement* *Metric* *Runner* *Mean previous week* *Mean last week* *Diff
> %* *Dashboard*
> go_batch_gbk_7 dataflow_runtime - 125.77 111.91 -11.02 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_pardo_2 dataflow_runtime - 420.66 360.73 -14.25 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_cogbk_2 dataflow_v2_java11_runtime_sec - 50.65 44.37 -12.41 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_gbk_3 dataflow_runtime_sec - 11.22 10.07 -10.3 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_6 dataflow_v2_java17_runtime_sec - 31.33 26.3 -16.06 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_runtime_sec - 49.25 44.03 -10.62 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_v2_java11_runtime_sec - 42.23 34.58 -18.11 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_pardo_1 sparkstructuredstreaming_runtime_sec - 56.17 49.55
> -11.8 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_runtime_sec - 13.8 12.29 -10.98 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_cogbk_1 dataflow_v2_java11_runtime_sec - 3095.03 2628.35
> -15.08 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_gbk_1 dataflow_v2_java17_runtime_sec - 1003.04 897.89
> -10.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_2 dataflow_v2_java11_runtime_sec - 4656.02 2798.25
> -39.9 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_3 dataflow_v2_java17_runtime_sec - 69.2 53.75 -22.32
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_5 dataflow_v2_java17_runtime_sec - 677.63 555.04 -18.09
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_pardo_1 dataflow_v2_java11_runtime_sec - 28.86 23.28 -19.35
> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_4 dataflow_v2_java17_runtime_sec - 37.98 33.86 -10.85
> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> kafkaioit_results read_time - 777.55 0.0 -100.0 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results write_time - 212.65 14.37 -93.24 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results_sdf_wrapper write_time - 6.25 3.97 -36.43 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> python_batch_cogbk_1 python_dataflow_batch_cogbk_1_runtime - 525.29 453.8
> -13.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_cogbk_3 python_dataflow_batch_cogbk_3_runtime - 282.71 253.8
> -10.23 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_combine_4 python_dataflow_batch_combine_4_runtime - 61.86
> 53.0 -14.32 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> python_batch_sideinput_2 python_dataflow_batch_sideinput_2_runtime - 50.14
> 45.0 -10.26 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_6 python_dataflow_batch_sideinput_6_runtime - 45.29
> 38.0 -16.09 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_bqio_write_10GB_results bqio_write_10GB_results_runtime - 247.86
> 210.2 -15.19 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
> python_streaming_gbk_3 python_dataflow_streaming_gbk_3_runtime - 90.86
> 78.5 -13.6 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> nexmark_11_batch RuntimeMs SparkStructuredStreamingRunner 1135.95 1015.14
> -10.64 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_sql_streaming RuntimeMs DataflowRunner 42997.25 34577.23 -19.58
> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_streaming RuntimeMs DataflowRunner 28812.61 24063.0 -16.48 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_15_batch RuntimeMs FlinkRunner 646.39 579.05 -10.42 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_2_batch RuntimeMs FlinkRunner 219.18 192.68 -12.09 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz> All metrics
> *Measurement* *Metric* *Runner* *Mean previous week* *Mean last week* *Diff
> %* *Dashboard*
> avroioit_hdfs_results read_time - 88.5 90.3 2.04 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> avroioit_hdfs_results write_time - 117.12 121.15 3.43 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> avroioit_results read_time - 90.57 93.45 3.18 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> avroioit_results write_time - 116.22 119.98 3.23 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> bqio_10GB_results_java_batch_avro avro_write_time - 35.46 34.18 -3.61 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> bqio_10GB_results_java_batch_avro read_time - 41.22 41.64 1.02 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> bqio_10GB_results_java_batch_json read_time - 42.36 38.94 -8.07 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> bqio_10GB_results_java_batch_json write_time - 51.59 50.42 -2.26 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> compressed_textioit_hdfs_results read_time - 158.32 160.91 1.64 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> compressed_textioit_hdfs_results write_time - 207.37 207.08 -0.14 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> compressed_textioit_results read_time - 158.41 161.48 1.94 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> compressed_textioit_results write_time - 204.95 204.34 -0.3 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> go_batch_cogbk_1 dataflow_runtime - 256.19 251.49 -1.84 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> go_batch_cogbk_2 dataflow_runtime - 148.89 143.6 -3.55 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> go_batch_cogbk_3 dataflow_runtime - 115.23 112.94 -1.99 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> go_batch_cogbk_4 dataflow_runtime - 114.24 112.66 -1.38 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> go_batch_combine_1 dataflow_runtime - 388.21 377.26 -2.82 [1]
> <http://metrics.beam.apache.org/d/WNzYt13Zk>
> go_batch_combine_4 dataflow_runtime - 77.88 76.26 -2.09 [1]
> <http://metrics.beam.apache.org/d/WNzYt13Zk>
> go_batch_combine_5 dataflow_runtime - 72.59 71.3 -1.78 [1]
> <http://metrics.beam.apache.org/d/WNzYt13Zk>
> go_batch_gbk_1 dataflow_runtime - 371.28 369.86 -0.38 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_2 dataflow_runtime - 100.97 98.34 -2.6 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_3 dataflow_runtime - 31.82 30.17 -5.18 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_4 dataflow_runtime - 55.12 54.68 -0.78 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_5 dataflow_runtime - 45.18 45.82 1.42 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_6 dataflow_runtime - 102.59 103.03 0.44 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_7 dataflow_runtime - 125.77 111.91 -11.02 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_pardo_1 dataflow_runtime - 75.89 78.22 3.07 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> go_batch_pardo_2 dataflow_runtime - 420.66 360.73 -14.25 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> go_batch_pardo_3 dataflow_runtime - 63.86 65.18 2.07 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> go_batch_pardo_4 dataflow_runtime - 72.36 70.43 -2.67 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> go_batch_sideinput_3 dataflow_runtime - 2.35 2.44 3.86 [1]
> <http://metrics.beam.apache.org/d/-E9aGlFGk>
> go_batch_sideinput_4 dataflow_runtime - 208.13 206.1 -0.98 [1]
> <http://metrics.beam.apache.org/d/-E9aGlFGk>
> hadoopformatioit_results read_time - 6.33 6.34 0.13 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> hadoopformatioit_results write_time - 7.79 7.51 -3.63 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> java_batch_cogbk_1 dataflow_runtime_sec - 41.75 42.28 1.27 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_1 dataflow_v2_java11_runtime_sec - 46.39 43.44 -6.36 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_1 dataflow_v2_java17_runtime_sec - 49.57 45.83 -7.54 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_1 sparkstructuredstreaming_runtime_sec - 118.05 124.41
> 5.39 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_2 dataflow_runtime_sec - 39.63 42.27 6.67 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_2 dataflow_v2_java11_runtime_sec - 50.65 44.37 -12.41 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_2 dataflow_v2_java17_runtime_sec - 44.58 44.66 0.18 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_2 sparkstructuredstreaming_runtime_sec - 116.79 122.12
> 4.56 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_3 dataflow_runtime_sec - 15.17 15.82 4.32 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_3 dataflow_v2_java11_runtime_sec - 14.41 13.74 -4.68 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_3 dataflow_v2_java17_runtime_sec - 13.58 16.39 20.7 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_3 sparkstructuredstreaming_runtime_sec - 19.96 20.61 3.28
> [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_4 dataflow_runtime_sec - 13.12 13.65 4.05 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_4 dataflow_v2_java11_runtime_sec - 14.25 15.78 10.76 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_4 dataflow_v2_java17_runtime_sec - 13.23 13.71 3.59 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_4 sparkstructuredstreaming_runtime_sec - 20.37 20.12
> -1.19 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_combine_1 dataflow_runtime_sec - 64.0 62.62 -2.16 [1]
> <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_combine_1 sparkstructuredstreaming_runtime_sec - 554.59 557.39
> 0.5 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_combine_4 dataflow_runtime_sec - 27.99 27.76 -0.8 [1]
> <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_combine_4 sparkstructuredstreaming_runtime_sec - 124.67 126.83
> 1.73 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_combine_5 dataflow_runtime_sec - 27.27 24.84 -8.93 [1]
> <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_combine_5 sparkstructuredstreaming_runtime_sec - 119.92 120.06
> 0.12 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_gbk_1 dataflow_runtime_sec - 125.42 115.83 -7.64 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_1 dataflow_v2_java11_runtime_sec - 96.68 95.35 -1.38 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_1 dataflow_v2_java17_runtime_sec - 86.51 90.35 4.44 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_1 sparkstructuredstreaming_runtime_sec - 502.91 503.81 0.18
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_2 dataflow_runtime_sec - 33.68 32.33 -4.0 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_2 dataflow_v2_java11_runtime_sec - 42.27 40.93 -3.18 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_2 dataflow_v2_java17_runtime_sec - 39.31 40.94 4.12 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_2 sparkstructuredstreaming_runtime_sec - 97.8 107.11 9.52
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_3 dataflow_runtime_sec - 11.22 10.07 -10.3 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_3 dataflow_v2_java11_runtime_sec - 14.44 17.31 19.88 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_3 dataflow_v2_java17_runtime_sec - 15.28 15.39 0.75 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_3 sparkstructuredstreaming_runtime_sec - 8.2 8.74 6.56 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_4 dataflow_runtime_sec - 18.85 19.23 2.05 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_4 dataflow_v2_java11_runtime_sec - 22.58 22.18 -1.78 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_4 dataflow_v2_java17_runtime_sec - 23.02 24.62 6.92 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_4 sparkstructuredstreaming_runtime_sec - 97.45 101.76 4.43
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_5 dataflow_runtime_sec - 18.5 17.83 -3.64 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_5 dataflow_v2_java11_runtime_sec - 17.53 17.45 -0.49 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_5 dataflow_v2_java17_runtime_sec - 19.43 20.54 5.68 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_5 sparkstructuredstreaming_runtime_sec - 100.52 100.52 -0.0
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_6 dataflow_runtime_sec - 37.2 34.88 -6.26 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_6 dataflow_v2_java11_runtime_sec - 26.4 26.36 -0.12 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_6 dataflow_v2_java17_runtime_sec - 31.33 26.3 -16.06 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_6 sparkstructuredstreaming_runtime_sec - 58.34 60.86 4.32
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_runtime_sec - 49.25 44.03 -10.62 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_v2_java11_runtime_sec - 42.23 34.58 -18.11 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_v2_java17_runtime_sec - 35.86 39.61 10.45 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 sparkstructuredstreaming_runtime_sec - 84.12 92.67 10.17
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_pardo_1 dataflow_runtime_sec - 20.04 18.14 -9.49 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_1 dataflow_v2_java11_runtime_sec - 13.25 16.08 21.34 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_1 dataflow_v2_java17_runtime_sec - 15.09 16.7 10.66 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_1 sparkstructuredstreaming_runtime_sec - 56.17 49.55
> -11.8 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_2 dataflow_runtime_sec - 186.08 202.59 8.87 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_2 dataflow_v2_java11_runtime_sec - 63.83 71.69 12.31 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_2 dataflow_v2_java17_runtime_sec - 66.38 67.13 1.13 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_runtime_sec - 13.8 12.29 -10.98 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_v2_java11_runtime_sec - 9.09 15.75 73.32 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_v2_java17_runtime_sec - 12.12 14.5 19.67 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_4 dataflow_runtime_sec - 24.8 22.52 -9.18 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_4 dataflow_v2_java11_runtime_sec - 26.58 24.46 -7.98 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_4 dataflow_v2_java17_runtime_sec - 25.41 24.93 -1.9 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_cogbk_1 dataflow_runtime_sec - 927.84 858.74 -7.45 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_1 dataflow_v2_java11_runtime_sec - 3095.03 2628.35
> -15.08 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_1 dataflow_v2_java17_runtime_sec - 2589.29 2595.64
> 0.25 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_2 dataflow_runtime_sec - 907.41 837.58 -7.7 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_2 dataflow_v2_java11_runtime_sec - 3256.56 2969.53
> -8.81 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_2 dataflow_v2_java17_runtime_sec - 2938.01 3090.66
> 5.2 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_3 dataflow_runtime_sec - 118.04 122.42 3.71 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_3 dataflow_v2_java11_runtime_sec - 322.79 326.12 1.03
> [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_3 dataflow_v2_java17_runtime_sec - 319.04 325.44 2.01
> [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_4 dataflow_runtime_sec - 125.95 122.06 -3.09 [1]
> <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_4 dataflow_v2_java11_runtime_sec - 320.92 326.47 1.73
> [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_4 dataflow_v2_java17_runtime_sec - 331.9 322.66 -2.78
> [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_combine_1 dataflow_runtime_sec - 509.03 511.79 0.54 [1]
> <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_streaming_combine_4 dataflow_runtime_sec - 318.94 322.22 1.03 [1]
> <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_streaming_combine_5 dataflow_runtime_sec - 307.67 309.5 0.6 [1]
> <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_streaming_gbk_1 dataflow_runtime_sec - 670.68 646.17 -3.65 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_1 dataflow_v2_java11_runtime_sec - 875.0 920.0 5.14 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_1 dataflow_v2_java17_runtime_sec - 1003.04 897.89
> -10.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_2 dataflow_runtime_sec - 803.33 798.42 -0.61 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_2 dataflow_v2_java11_runtime_sec - 4656.02 2798.25
> -39.9 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_2 dataflow_v2_java17_runtime_sec - 3158.86 3166.16 0.23
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_3 dataflow_runtime_sec - 23.62 24.92 5.49 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_3 dataflow_v2_java11_runtime_sec - 53.16 59.27 11.48
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_3 dataflow_v2_java17_runtime_sec - 69.2 53.75 -22.32
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_4 dataflow_runtime_sec - 289.5 286.49 -1.04 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_4 dataflow_v2_java11_runtime_sec - 641.36 649.91 1.33
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_4 dataflow_v2_java17_runtime_sec - 665.68 685.61 2.99
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_5 dataflow_runtime_sec - 293.31 280.95 -4.21 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_5 dataflow_v2_java11_runtime_sec - 602.18 632.83 5.09
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_5 dataflow_v2_java17_runtime_sec - 677.63 555.04 -18.09
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_6 dataflow_runtime_sec - 73.86 76.09 3.02 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_6 dataflow_v2_java11_runtime_sec - 160.18 145.57 -9.12
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_6 dataflow_v2_java17_runtime_sec - 194.24 178.72 -7.99
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_7 dataflow_runtime_sec - 137.68 137.62 -0.04 [1]
> <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_7 dataflow_v2_java11_runtime_sec - 207.62 226.19 8.94
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_7 dataflow_v2_java17_runtime_sec - 223.14 202.22 -9.37
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_pardo_1 dataflow_runtime_sec - 18.79 18.93 0.75 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_1 dataflow_v2_java11_runtime_sec - 28.86 23.28 -19.35
> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_1 dataflow_v2_java17_runtime_sec - 25.27 24.23 -4.08
> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_2 dataflow_runtime_sec - 223.51 226.23 1.22 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_2 dataflow_v2_java11_runtime_sec - 114.31 111.55
> -2.41 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_2 dataflow_v2_java17_runtime_sec - 105.51 97.51 -7.58
> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_3 dataflow_runtime_sec - 12.55 12.71 1.25 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_3 dataflow_v2_java11_runtime_sec - 21.3 23.56 10.62
> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_3 dataflow_v2_java17_runtime_sec - 24.03 24.17 0.61
> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_4 dataflow_runtime_sec - 32.98 33.47 1.49 [1]
> <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_4 dataflow_v2_java11_runtime_sec - 36.78 35.46 -3.61
> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_4 dataflow_v2_java17_runtime_sec - 37.98 33.86 -10.85
> [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> jdbcioit_results read_time - 24.89 24.82 -0.28 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> jdbcioit_results write_time - 38.16 36.52 -4.3 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results read_time - 777.55 0.0 -100.0 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results write_time - 212.65 14.37 -93.24 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results_runner_v2 read_time - 1.87 2.36 26.33 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results_runner_v2 write_time - 5.89 5.91 0.4 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results_sdf_wrapper read_time - 3.04 3.26 7.08 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results_sdf_wrapper write_time - 6.25 3.97 -36.43 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> many_files_textioit_hdfs_results read_time - 17.29 17.48 1.13 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> many_files_textioit_hdfs_results write_time - 61.16 56.82 -7.09 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> many_files_textioit_results read_time - 24.9 24.23 -2.67 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> many_files_textioit_results write_time - 61.41 58.16 -5.29 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> mongodbioit_results read_time - 30.18 30.83 2.15 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> mongodbioit_results write_time - 59.02 55.45 -6.06 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> parquetioit_hdfs_results read_time - 88.2 88.97 0.88 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> parquetioit_hdfs_results write_time - 147.1 142.12 -3.39 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> parquetioit_results read_time - 87.88 89.9 2.29 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> parquetioit_results write_time - 145.06 139.23 -4.02 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> python_batch_cogbk_1 python_dataflow_batch_cogbk_1_runtime - 525.29 453.8
> -13.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_cogbk_2 python_dataflow_batch_cogbk_2_runtime - 287.43 317.2
> 10.36 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_cogbk_3 python_dataflow_batch_cogbk_3_runtime - 282.71 253.8
> -10.23 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_cogbk_4 python_dataflow_batch_cogbk_4_runtime - 291.71 278.4
> -4.56 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_combine_1 python_dataflow_batch_combine_1_runtime - 841.29
> 875.6 4.08 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> python_batch_combine_4 python_dataflow_batch_combine_4_runtime - 61.86
> 53.0 -14.32 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> python_batch_combine_5 python_dataflow_batch_combine_5_runtime - 46.0 44.4
> -3.48 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> python_batch_gbk_1 python_dataflow_batch_gbk_1_runtime - 1877.29 1915.8
> 2.05 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_2 python_dataflow_batch_gbk_2_runtime - 373.14 367.8
> -1.43 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_3 python_dataflow_batch_gbk_3_runtime - 27.43 32.6 18.85
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_4 python_dataflow_batch_gbk_4_runtime - 124.43 124.2
> -0.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_5 python_dataflow_batch_gbk_5_runtime - 127.0 129.4 1.89
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_6 python_dataflow_batch_gbk_6_runtime - 344.86 352.4 2.19
> [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_7 python_dataflow_batch_gbk_7_runtime - 353.14 346.0
> -2.02 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_pardo_1 python_dataflow_batch_pardo_1_runtime - 180.14 176.8
> -1.86 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_batch_pardo_2 python_dataflow_batch_pardo_2_runtime - 1635.57
> 1661.4 1.58 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_batch_pardo_3 python_dataflow_batch_pardo_3_runtime - 141.86 150.6
> 6.16 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_batch_pardo_4 python_dataflow_batch_pardo_4_runtime - 326.71 345.2
> 5.66 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_batch_sideinput_1 python_dataflow_batch_sideinput_1_runtime - 47.86
> 46.4 -3.04 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_2 python_dataflow_batch_sideinput_2_runtime - 50.14
> 45.0 -10.26 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_3 python_dataflow_batch_sideinput_3_runtime - 5.71
> 5.4 -5.5 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_4 python_dataflow_batch_sideinput_4_runtime -
> 471.29 436.4 -7.4 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_5 python_dataflow_batch_sideinput_5_runtime - 47.29
> 44.4 -6.1 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_6 python_dataflow_batch_sideinput_6_runtime - 45.29
> 38.0 -16.09 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_7 python_dataflow_batch_sideinput_7_runtime -
> 5566.29 5588.2 0.39 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_8 python_dataflow_batch_sideinput_8_runtime -
> 5592.14 5562.8 -0.52 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_bqio_read_10GB_results bqio_read_10GB_results_runtime - 118.71
> 111.6 -5.99 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
> python_bqio_write_10GB_results bqio_write_10GB_results_runtime - 247.86
> 210.2 -15.19 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
> python_direct_microbenchmarks fn_api_runner_microbenchmark_runtime_sec -
> 94.83 94.11 -0.76
> python_direct_microbenchmarks teststream_microbenchmark_runtime_sec -
> 344.58 337.86 -1.95
> python_spanner_write_2GB_results pyspanner_write_2GB_results_runtime -
> 1562.57 1599.2 2.34 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
> python_spannerio_read_2GB_results pyspanner_read_2GB_results_runtime -
> 26.43 23.8 -9.95 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
> python_streaming_cogbk_1 python_dataflow_streaming_cogbk_1_runtime -
> 628.29 659.8 5.02 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_streaming_cogbk_2 python_dataflow_streaming_cogbk_2_runtime - 425.0
> 422.4 -0.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_streaming_cogbk_3 python_dataflow_streaming_cogbk_3_runtime -
> 506.43 491.2 -3.01 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_streaming_cogbk_4 python_dataflow_streaming_cogbk_4_runtime -
> 424.71 427.0 0.54 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_streaming_gbk_3 python_dataflow_streaming_gbk_3_runtime - 90.86
> 78.5 -13.6 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_streaming_gbk_6 python_dataflow_streaming_gbk_6_runtime - 523.57
> 546.0 4.28 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_streaming_gbk_7 python_dataflow_streaming_gbk_7_runtime - 506.29
> 553.5 9.33 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_streaming_pardo_1 python_dataflow_streaming_pardo_1_runtime -
> 253.57 235.5 -7.13 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_streaming_pardo_2 python_dataflow_streaming_pardo_2_runtime -
> 2941.57 3096.5 5.27 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_streaming_pardo_3 python_dataflow_streaming_pardo_3_runtime - 204.0
> 200.0 -1.96 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_streaming_pardo_4 python_dataflow_streaming_pardo_4_runtime -
> 538.57 602.0 11.78 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> textioit_hdfs_results read_time - 19.88 20.73 4.27 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> textioit_hdfs_results write_time - 32.34 31.18 -3.59 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> textioit_results read_time - 20.52 21.58 5.16 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> textioit_results write_time - 32.12 32.81 2.14 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> tfrecordioit_results read_time - 15.43 16.26 5.35 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> tfrecordioit_results write_time - 20.79 21.14 1.68 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> wordcount_py37_results runtime - 811.08 824.06 1.6 [1]
> <http://metrics.beam.apache.org/d/1cnwVDkGk>
> xmlioit_hdfs_results read_time - 21.4 22.77 6.44 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> xmlioit_hdfs_results write_time - 24.67 25.12 1.79 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> xmlioit_results read_time - 23.48 23.36 -0.53 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> xmlioit_results write_time - 24.44 26.24 7.36 [1]
> <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> nexmark_0_batch RuntimeMs DataflowRunner 27146.02 27212.08 0.24 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_batch RuntimeMs DirectRunner 4782.36 4783.0 0.01 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_batch RuntimeMs FlinkRunner 1155.07 1111.86 -3.74 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_0_batch RuntimeMs SparkRunner 1253.64 1255.18 0.12 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_batch RuntimeMs SparkStructuredStreamingRunner 1546.91 1554.18
> 0.47 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_0_sql_batch RuntimeMs DataflowRunner 35274.21 35610.5 0.95 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_0_sql_batch RuntimeMs DirectRunner 5570.26 5655.59 1.53 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_sql_batch RuntimeMs SparkRunner 3205.2 3141.48 -1.99 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_0_sql_batch RuntimeMs SparkStructuredStreamingRunner 2939.46
> 2926.11 -0.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_0_sql_streaming RuntimeMs DataflowRunner 24026.89 23121.55 -3.77
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_sql_streaming RuntimeMs DirectRunner 15501.0 15555.91 0.35 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_sql_streaming RuntimeMs FlinkRunner 1637.46 1628.64 -0.54 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_0_streaming RuntimeMs DataflowRunner 39291.92 37816.14 -3.76 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_streaming RuntimeMs DirectRunner 11811.79 11938.45 1.07 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_streaming RuntimeMs FlinkRunner 1432.14 1422.36 -0.68 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_0_zetasql_batch RuntimeMs DataflowRunner 114915.96 112238.45 -2.33
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_0_zetasql_batch RuntimeMs DirectRunner 6757.07 6834.27 1.14 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_0_zetasql_streaming RuntimeMs DataflowRunner 67751.04 66627.91
> -1.66 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_zetasql_streaming RuntimeMs DirectRunner 17105.75 16996.45 -0.64
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_zetasql_streaming RuntimeMs FlinkRunner 3248.54 3261.41 0.4 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_10_batch RuntimeMs DataflowRunner 48690.3 49306.87 1.27 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_10_batch RuntimeMs DirectRunner 10463.32 10465.09 0.02 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_10_batch RuntimeMs FlinkRunner 777.21 714.64 -8.05 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_10_batch RuntimeMs SparkRunner 726.5 729.61 0.43 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_10_batch RuntimeMs SparkStructuredStreamingRunner 1126.52 1211.0
> 7.5 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_10_streaming RuntimeMs DataflowRunner 105776.61 102899.66 -2.72
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_10_streaming RuntimeMs DirectRunner 43926.21 44091.14 0.38 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_10_streaming RuntimeMs FlinkRunner 1535.29 1488.14 -3.07 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_11_batch RuntimeMs DataflowRunner 39589.51 61325.57 54.9 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_11_batch RuntimeMs DirectRunner 3757.79 3770.55 0.34 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_11_batch RuntimeMs FlinkRunner 297.43 313.64 5.45 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_11_batch RuntimeMs SparkRunner 1402.09 1399.52 -0.18 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_11_batch RuntimeMs SparkStructuredStreamingRunner 1135.95 1015.14
> -10.64 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_11_streaming RuntimeMs DataflowRunner 182507.05 181193.81 -0.72
> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_11_streaming RuntimeMs DirectRunner 10696.0 10739.0 0.4 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_11_streaming RuntimeMs FlinkRunner 1351.79 1319.05 -2.42 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_12_batch RuntimeMs DataflowRunner 19630.69 19666.75 0.18 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_12_batch RuntimeMs DirectRunner 2597.07 2626.36 1.13 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_12_batch RuntimeMs FlinkRunner 180.29 173.05 -4.02 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_12_batch RuntimeMs SparkRunner 924.52 929.25 0.51 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_12_batch RuntimeMs SparkStructuredStreamingRunner 827.98 899.98
> 8.7 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_12_streaming RuntimeMs DataflowRunner 96217.51 94780.91 -1.49 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_12_streaming RuntimeMs DirectRunner 9335.68 9409.5 0.79 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_12_streaming RuntimeMs FlinkRunner 621.5 620.73 -0.12 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_batch RuntimeMs DataflowRunner 33870.18 33840.36 -0.09 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_batch RuntimeMs DirectRunner 3225.36 3231.86 0.2 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_13_batch RuntimeMs FlinkRunner 392.68 404.95 3.13 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_13_batch RuntimeMs SparkRunner 355.27 357.02 0.49 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_batch RuntimeMs SparkStructuredStreamingRunner 547.32 556.57
> 1.69 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_13_sql_batch RuntimeMs DataflowRunner 62806.75 61308.68 -2.39 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_13_sql_batch RuntimeMs DirectRunner 9065.15 9052.5 -0.14 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_sql_batch RuntimeMs SparkRunner 3473.45 3430.11 -1.25 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_sql_batch RuntimeMs SparkStructuredStreamingRunner 3113.52
> 3060.59 -1.7 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_sql_streaming RuntimeMs DataflowRunner 42997.25 34577.23 -19.58
> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_sql_streaming RuntimeMs DirectRunner 18918.29 18631.09 -1.52
> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_sql_streaming RuntimeMs FlinkRunner 28617.96 29781.77 4.07 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_streaming RuntimeMs DataflowRunner 28812.61 24063.0 -16.48 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_streaming RuntimeMs DirectRunner 9996.14 9949.18 -0.47 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_streaming RuntimeMs FlinkRunner 27765.36 28396.95 2.27 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_zetasql_batch RuntimeMs DataflowRunner 262683.96 259338.23
> -1.27 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_13_zetasql_batch RuntimeMs DirectRunner 10046.04 10059.45 0.13 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_zetasql_streaming RuntimeMs DataflowRunner 116689.18 109084.86
> -6.52 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_zetasql_streaming RuntimeMs DirectRunner 21036.07 21342.05 1.45
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_zetasql_streaming RuntimeMs FlinkRunner 34089.0 34659.64 1.67
> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_14_batch RuntimeMs DataflowRunner 62350.36 61308.36 -1.67 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_14_batch RuntimeMs DirectRunner 4153.39 4150.05 -0.08 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_14_batch RuntimeMs FlinkRunner 666.86 650.5 -2.45 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_14_batch RuntimeMs SparkRunner 895.64 945.5 5.57 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_14_batch RuntimeMs SparkStructuredStreamingRunner 933.43 941.27
> 0.84 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_14_streaming RuntimeMs DataflowRunner 121083.79 126024.55 4.08 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_14_streaming RuntimeMs DirectRunner 12338.79 12202.36 -1.11 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_14_streaming RuntimeMs FlinkRunner 1783.46 1762.27 -1.19 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_15_batch RuntimeMs DataflowRunner 45464.97 45823.83 0.79 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_15_batch RuntimeMs DirectRunner 15421.04 15413.5 -0.05 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_15_batch RuntimeMs FlinkRunner 646.39 579.05 -10.42 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_15_batch RuntimeMs SparkRunner 1643.96 1675.16 1.9 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_15_batch RuntimeMs SparkStructuredStreamingRunner 1451.93 1442.11
> -0.68 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_16_batch RuntimeMs DataflowRunner 51963.45 51419.82 -1.05 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_16_batch RuntimeMs DirectRunner 85652.86 84922.68 -0.85 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_16_batch RuntimeMs FlinkRunner 214.46 256.23 19.47 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_16_batch RuntimeMs SparkRunner 1484.05 1386.3 -6.59 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_16_batch RuntimeMs SparkStructuredStreamingRunner 1249.75 1199.98
> -3.98 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_16_streaming RuntimeMs DataflowRunner 89716.22 95450.93 6.39 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_16_streaming RuntimeMs DirectRunner 124442.11 129482.0 4.05 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_16_streaming RuntimeMs FlinkRunner 423.43 431.27 1.85 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_batch RuntimeMs DataflowRunner 21877.32 20846.38 -4.71 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_batch RuntimeMs DirectRunner 3502.18 3394.41 -3.08 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_1_batch RuntimeMs FlinkRunner 315.89 307.91 -2.53 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_1_batch RuntimeMs SparkRunner 440.91 442.09 0.27 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_1_batch RuntimeMs SparkStructuredStreamingRunner 591.54 592.73 0.2
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_1_sql_batch RuntimeMs DataflowRunner 32344.21 32125.86 -0.68 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_1_sql_batch RuntimeMs DirectRunner 3546.3 3726.64 5.09 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_sql_batch RuntimeMs SparkRunner 2430.38 2390.05 -1.66 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_1_sql_batch RuntimeMs SparkStructuredStreamingRunner 1572.41
> 1666.39 5.98 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_1_sql_streaming RuntimeMs DataflowRunner 20870.43 21097.45 1.09
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_1_sql_streaming RuntimeMs DirectRunner 11466.04 11491.86 0.23 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_sql_streaming RuntimeMs FlinkRunner 667.54 637.77 -4.46 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_streaming RuntimeMs DataflowRunner 28937.91 28601.91 -1.16 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_streaming RuntimeMs DirectRunner 8206.14 8198.05 -0.1 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_streaming RuntimeMs FlinkRunner 648.86 667.5 2.87 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_batch RuntimeMs DataflowRunner 15263.96 15124.28 -0.92 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_2_batch RuntimeMs DirectRunner 1530.89 1605.64 4.88 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_2_batch RuntimeMs FlinkRunner 219.18 192.68 -12.09 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_batch RuntimeMs SparkRunner 311.59 310.82 -0.25 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_batch RuntimeMs SparkStructuredStreamingRunner 458.84 460.77
> 0.42 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_2_sql_batch RuntimeMs DataflowRunner 25789.39 25085.68 -2.73 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_2_sql_batch RuntimeMs DirectRunner 1543.37 1555.64 0.79 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_2_sql_batch RuntimeMs SparkRunner 744.41 744.02 -0.05 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_sql_batch RuntimeMs SparkStructuredStreamingRunner 1005.04
> 972.02 -3.28 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_2_sql_streaming RuntimeMs DataflowRunner 13841.29 13565.55 -1.99
> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_sql_streaming RuntimeMs DirectRunner 6789.04 6801.5 0.18 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_sql_streaming RuntimeMs FlinkRunner 424.61 410.73 -3.27 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_streaming RuntimeMs DataflowRunner 19257.58 18899.86 -1.86 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_streaming RuntimeMs DirectRunner 5142.04 5144.05 0.04 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_streaming RuntimeMs FlinkRunner 270.54 275.0 1.65 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_zetasql_batch RuntimeMs DataflowRunner 71538.04 73956.5 3.38 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_zetasql_batch RuntimeMs DirectRunner 2908.36 2883.55 -0.85 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_zetasql_streaming RuntimeMs DataflowRunner 39600.89 40732.09
> 2.86 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_zetasql_streaming RuntimeMs DirectRunner 8218.75 8148.45 -0.86
> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_zetasql_streaming RuntimeMs FlinkRunner 1202.64 1165.27 -3.11
> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_batch RuntimeMs DataflowRunner 26748.14 27266.41 1.94 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_batch RuntimeMs DirectRunner 3224.14 3243.5 0.6 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_batch RuntimeMs FlinkRunner 1105.75 1059.41 -4.19 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_3_batch RuntimeMs SparkRunner 2086.2 2081.61 -0.22 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_sql_batch RuntimeMs DataflowRunner 29239.46 29789.05 1.88 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_3_sql_batch RuntimeMs DirectRunner 3365.15 3329.5 -1.06 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_3_sql_batch RuntimeMs SparkRunner 3503.88 3480.3 -0.67 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_3_sql_batch RuntimeMs SparkStructuredStreamingRunner 3032.75
> 2996.09 -1.21 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_3_sql_streaming RuntimeMs DataflowRunner 33163.79 33241.18 0.23
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_sql_streaming RuntimeMs DirectRunner 8460.71 8474.64 0.16 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_sql_streaming RuntimeMs FlinkRunner 970.11 960.36 -1.0 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_streaming RuntimeMs DataflowRunner 43269.14 43554.49 0.66 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_streaming RuntimeMs DirectRunner 7623.46 7534.73 -1.16 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_streaming RuntimeMs FlinkRunner 469.46 456.77 -2.7 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_zetasql_batch RuntimeMs DataflowRunner 54330.46 54539.23 0.38
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_3_zetasql_batch RuntimeMs DirectRunner 3749.36 3745.59 -0.1 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_zetasql_streaming RuntimeMs DataflowRunner 46361.32 46243.59
> -0.25 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_zetasql_streaming RuntimeMs DirectRunner 8915.93 8861.45 -0.61
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_zetasql_streaming RuntimeMs FlinkRunner 1191.43 1197.64 0.52 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_4_batch RuntimeMs DataflowRunner 33212.25 33292.5 0.24 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_4_batch RuntimeMs DirectRunner 2416.46 2409.05 -0.31 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_4_batch RuntimeMs FlinkRunner 861.68 851.14 -1.22 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_4_batch RuntimeMs SparkRunner 1001.52 998.98 -0.25 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_4_batch RuntimeMs SparkStructuredStreamingRunner 2243.59 2233.52
> -0.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_4_streaming RuntimeMs DataflowRunner 49175.82 49133.95 -0.09 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_4_streaming RuntimeMs DirectRunner 2827.5 2862.05 1.22 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_4_streaming RuntimeMs FlinkRunner 975.32 1021.45 4.73 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_5_batch RuntimeMs DataflowRunner 35151.96 34758.3 -1.12 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_5_batch RuntimeMs DirectRunner 10582.36 10621.95 0.37 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_5_batch RuntimeMs FlinkRunner 775.96 769.41 -0.84 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_5_batch RuntimeMs SparkRunner 1515.27 1535.14 1.31 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_5_batch RuntimeMs SparkStructuredStreamingRunner 1719.73 1732.2
> 0.73 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_5_streaming RuntimeMs DataflowRunner 214447.03 209021.26 -2.53 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_5_streaming RuntimeMs DirectRunner 20728.25 21227.32 2.41 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_5_streaming RuntimeMs FlinkRunner 1609.86 1637.59 1.72 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_6_batch RuntimeMs DataflowRunner 31298.14 31001.5 -0.95 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_6_batch RuntimeMs DirectRunner 1886.82 1851.91 -1.85 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_6_batch RuntimeMs FlinkRunner 536.79 519.91 -3.14 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_6_batch RuntimeMs SparkRunner 710.09 706.14 -0.56 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_6_batch RuntimeMs SparkStructuredStreamingRunner 907.57 948.52
> 4.51 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_6_streaming RuntimeMs DataflowRunner 38287.43 41809.0 9.2 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_6_streaming RuntimeMs DirectRunner 2135.57 2098.95 -1.71 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_6_streaming RuntimeMs FlinkRunner 601.57 592.18 -1.56 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_batch RuntimeMs DataflowRunner 54975.39 56478.73 2.73 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_7_batch RuntimeMs DirectRunner 18420.82 18547.23 0.69 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_7_batch RuntimeMs FlinkRunner 665.07 677.59 1.88 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_7_batch RuntimeMs SparkRunner 1440.04 1426.75 -0.92 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_7_batch RuntimeMs SparkStructuredStreamingRunner 2214.61 2196.27
> -0.83 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_7_sql_batch RuntimeMs DataflowRunner 78570.29 78734.55 0.21 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_7_sql_batch RuntimeMs DirectRunner 326788.15 364568.64 11.56 [1]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_sql_batch RuntimeMs SparkRunner 7212.82 7068.95 -1.99 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_sql_batch RuntimeMs SparkStructuredStreamingRunner 5996.48
> 5910.11 -1.44 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_7_sql_streaming RuntimeMs DataflowRunner 1003130.36 1027200.23 2.4
> [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_7_sql_streaming RuntimeMs DirectRunner 1076402.96 1215625.41 12.93
> [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_sql_streaming RuntimeMs FlinkRunner 3486.11 3459.86 -0.75 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_streaming RuntimeMs DataflowRunner 120673.19 122350.28 1.39 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_7_streaming RuntimeMs DirectRunner 850907.18 848503.82 -0.28 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_7_streaming RuntimeMs FlinkRunner 13410.82 13769.05 2.67 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_8_batch RuntimeMs DataflowRunner 21572.36 21636.01 0.3 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_8_batch RuntimeMs DirectRunner 2258.14 2383.27 5.54 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_8_batch RuntimeMs FlinkRunner 473.39 478.09 0.99 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_8_batch RuntimeMs SparkRunner 1451.93 1434.59 -1.19 [1]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_8_batch RuntimeMs SparkStructuredStreamingRunner 1185.75 1128.57
> -4.82 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_8_streaming RuntimeMs DataflowRunner 59645.41 61147.95 2.52 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_8_streaming RuntimeMs DirectRunner 6981.75 7187.59 2.95 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_8_streaming RuntimeMs FlinkRunner 479.46 478.59 -0.18 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_9_batch RuntimeMs DataflowRunner 29188.29 30279.18 3.74 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [3]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_9_batch RuntimeMs DirectRunner 1454.29 1440.0 -0.98 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_9_batch RuntimeMs FlinkRunner 290.18 325.45 12.16 [1]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_9_batch RuntimeMs SparkRunner 541.73 544.66 0.54 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/tkqc0AdGk> [4]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_9_batch RuntimeMs SparkStructuredStreamingRunner 708.32 716.98
> 1.22 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3]
> <http://metrics.beam.apache.org/d/8INnSY9Mz> [4]
> <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_9_streaming RuntimeMs DataflowRunner 31862.82 31803.18 -0.19 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_9_streaming RuntimeMs DirectRunner 1758.86 1774.09 0.87 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_9_streaming RuntimeMs FlinkRunner 430.46 487.23 13.19 [1]
> <http://metrics.beam.apache.org/d/ahudA_zGz> [2]
> <http://metrics.beam.apache.org/d/8INnSY9Mz>
>
>
>
Re: Beam Metrics Report (2022-07-12)
Posted by Alexey Romanenko <ar...@gmail.com>.
Taking a look on red possible regressions number, I notice that it can be caused by some outliers that may be happened because of, for example, resources issues on Jenkins.
(nexmark_11_batch, DataflowRunner)
Should we use a median value instead of a mean one to make it more representative? Thoughts?
—
Alexey
> On 12 Jul 2022, at 13:55, Apache Jenkins Server <je...@builds.apache.org> wrote:
>
> Beam Metrics Report
>
> Color legend:
> >= 20 >= 10 <= -10 <= -20
> Possible regression
>
> Measurement Metric Runner Mean previous week Mean last week Diff % Dashboard
> java_batch_cogbk_3 dataflow_v2_java17_runtime_sec - 13.58 16.39 20.7 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_4 dataflow_v2_java11_runtime_sec - 14.25 15.78 10.76 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_gbk_3 dataflow_v2_java11_runtime_sec - 14.44 17.31 19.88 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_v2_java17_runtime_sec - 35.86 39.61 10.45 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 sparkstructuredstreaming_runtime_sec - 84.12 92.67 10.17 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_pardo_1 dataflow_v2_java11_runtime_sec - 13.25 16.08 21.34 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_1 dataflow_v2_java17_runtime_sec - 15.09 16.7 10.66 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_2 dataflow_v2_java11_runtime_sec - 63.83 71.69 12.31 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_v2_java11_runtime_sec - 9.09 15.75 73.32 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_v2_java17_runtime_sec - 12.12 14.5 19.67 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_gbk_3 dataflow_v2_java11_runtime_sec - 53.16 59.27 11.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_pardo_3 dataflow_v2_java11_runtime_sec - 21.3 23.56 10.62 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> kafkaioit_results_runner_v2 read_time - 1.87 2.36 26.33 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> python_batch_cogbk_2 python_dataflow_batch_cogbk_2_runtime - 287.43 317.2 10.36 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_gbk_3 python_dataflow_batch_gbk_3_runtime - 27.43 32.6 18.85 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_streaming_pardo_4 python_dataflow_streaming_pardo_4_runtime - 538.57 602.0 11.78 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> nexmark_11_batch RuntimeMs DataflowRunner 39589.51 61325.57 54.9 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_16_batch RuntimeMs FlinkRunner 214.46 256.23 19.47 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_7_sql_batch RuntimeMs DirectRunner 326788.15 364568.64 11.56 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_sql_streaming RuntimeMs DirectRunner 1076402.96 1215625.41 12.93 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_9_batch RuntimeMs FlinkRunner 290.18 325.45 12.16 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_9_streaming RuntimeMs FlinkRunner 430.46 487.23 13.19 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> Possible improvement
>
> Measurement Metric Runner Mean previous week Mean last week Diff % Dashboard
> go_batch_gbk_7 dataflow_runtime - 125.77 111.91 -11.02 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_pardo_2 dataflow_runtime - 420.66 360.73 -14.25 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_cogbk_2 dataflow_v2_java11_runtime_sec - 50.65 44.37 -12.41 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_gbk_3 dataflow_runtime_sec - 11.22 10.07 -10.3 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_6 dataflow_v2_java17_runtime_sec - 31.33 26.3 -16.06 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_runtime_sec - 49.25 44.03 -10.62 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_v2_java11_runtime_sec - 42.23 34.58 -18.11 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_pardo_1 sparkstructuredstreaming_runtime_sec - 56.17 49.55 -11.8 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_runtime_sec - 13.8 12.29 -10.98 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_cogbk_1 dataflow_v2_java11_runtime_sec - 3095.03 2628.35 -15.08 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_gbk_1 dataflow_v2_java17_runtime_sec - 1003.04 897.89 -10.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_2 dataflow_v2_java11_runtime_sec - 4656.02 2798.25 -39.9 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_3 dataflow_v2_java17_runtime_sec - 69.2 53.75 -22.32 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_5 dataflow_v2_java17_runtime_sec - 677.63 555.04 -18.09 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_pardo_1 dataflow_v2_java11_runtime_sec - 28.86 23.28 -19.35 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_4 dataflow_v2_java17_runtime_sec - 37.98 33.86 -10.85 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> kafkaioit_results read_time - 777.55 0.0 -100.0 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results write_time - 212.65 14.37 -93.24 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results_sdf_wrapper write_time - 6.25 3.97 -36.43 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> python_batch_cogbk_1 python_dataflow_batch_cogbk_1_runtime - 525.29 453.8 -13.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_cogbk_3 python_dataflow_batch_cogbk_3_runtime - 282.71 253.8 -10.23 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_combine_4 python_dataflow_batch_combine_4_runtime - 61.86 53.0 -14.32 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> python_batch_sideinput_2 python_dataflow_batch_sideinput_2_runtime - 50.14 45.0 -10.26 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_6 python_dataflow_batch_sideinput_6_runtime - 45.29 38.0 -16.09 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_bqio_write_10GB_results bqio_write_10GB_results_runtime - 247.86 210.2 -15.19 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
> python_streaming_gbk_3 python_dataflow_streaming_gbk_3_runtime - 90.86 78.5 -13.6 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> nexmark_11_batch RuntimeMs SparkStructuredStreamingRunner 1135.95 1015.14 -10.64 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_sql_streaming RuntimeMs DataflowRunner 42997.25 34577.23 -19.58 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_streaming RuntimeMs DataflowRunner 28812.61 24063.0 -16.48 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_15_batch RuntimeMs FlinkRunner 646.39 579.05 -10.42 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_2_batch RuntimeMs FlinkRunner 219.18 192.68 -12.09 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> All metrics
>
> Measurement Metric Runner Mean previous week Mean last week Diff % Dashboard
> avroioit_hdfs_results read_time - 88.5 90.3 2.04 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> avroioit_hdfs_results write_time - 117.12 121.15 3.43 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> avroioit_results read_time - 90.57 93.45 3.18 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> avroioit_results write_time - 116.22 119.98 3.23 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> bqio_10GB_results_java_batch_avro avro_write_time - 35.46 34.18 -3.61 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> bqio_10GB_results_java_batch_avro read_time - 41.22 41.64 1.02 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> bqio_10GB_results_java_batch_json read_time - 42.36 38.94 -8.07 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> bqio_10GB_results_java_batch_json write_time - 51.59 50.42 -2.26 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> compressed_textioit_hdfs_results read_time - 158.32 160.91 1.64 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> compressed_textioit_hdfs_results write_time - 207.37 207.08 -0.14 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> compressed_textioit_results read_time - 158.41 161.48 1.94 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> compressed_textioit_results write_time - 204.95 204.34 -0.3 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> go_batch_cogbk_1 dataflow_runtime - 256.19 251.49 -1.84 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> go_batch_cogbk_2 dataflow_runtime - 148.89 143.6 -3.55 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> go_batch_cogbk_3 dataflow_runtime - 115.23 112.94 -1.99 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> go_batch_cogbk_4 dataflow_runtime - 114.24 112.66 -1.38 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> go_batch_combine_1 dataflow_runtime - 388.21 377.26 -2.82 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> go_batch_combine_4 dataflow_runtime - 77.88 76.26 -2.09 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> go_batch_combine_5 dataflow_runtime - 72.59 71.3 -1.78 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> go_batch_gbk_1 dataflow_runtime - 371.28 369.86 -0.38 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_2 dataflow_runtime - 100.97 98.34 -2.6 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_3 dataflow_runtime - 31.82 30.17 -5.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_4 dataflow_runtime - 55.12 54.68 -0.78 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_5 dataflow_runtime - 45.18 45.82 1.42 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_6 dataflow_runtime - 102.59 103.03 0.44 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_gbk_7 dataflow_runtime - 125.77 111.91 -11.02 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> go_batch_pardo_1 dataflow_runtime - 75.89 78.22 3.07 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> go_batch_pardo_2 dataflow_runtime - 420.66 360.73 -14.25 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> go_batch_pardo_3 dataflow_runtime - 63.86 65.18 2.07 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> go_batch_pardo_4 dataflow_runtime - 72.36 70.43 -2.67 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> go_batch_sideinput_3 dataflow_runtime - 2.35 2.44 3.86 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> go_batch_sideinput_4 dataflow_runtime - 208.13 206.1 -0.98 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> hadoopformatioit_results read_time - 6.33 6.34 0.13 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> hadoopformatioit_results write_time - 7.79 7.51 -3.63 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> java_batch_cogbk_1 dataflow_runtime_sec - 41.75 42.28 1.27 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_1 dataflow_v2_java11_runtime_sec - 46.39 43.44 -6.36 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_1 dataflow_v2_java17_runtime_sec - 49.57 45.83 -7.54 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_1 sparkstructuredstreaming_runtime_sec - 118.05 124.41 5.39 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_2 dataflow_runtime_sec - 39.63 42.27 6.67 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_2 dataflow_v2_java11_runtime_sec - 50.65 44.37 -12.41 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_2 dataflow_v2_java17_runtime_sec - 44.58 44.66 0.18 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_2 sparkstructuredstreaming_runtime_sec - 116.79 122.12 4.56 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_3 dataflow_runtime_sec - 15.17 15.82 4.32 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_3 dataflow_v2_java11_runtime_sec - 14.41 13.74 -4.68 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_3 dataflow_v2_java17_runtime_sec - 13.58 16.39 20.7 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_3 sparkstructuredstreaming_runtime_sec - 19.96 20.61 3.28 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_4 dataflow_runtime_sec - 13.12 13.65 4.05 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_4 dataflow_v2_java11_runtime_sec - 14.25 15.78 10.76 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_4 dataflow_v2_java17_runtime_sec - 13.23 13.71 3.59 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_cogbk_4 sparkstructuredstreaming_runtime_sec - 20.37 20.12 -1.19 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_batch_combine_1 dataflow_runtime_sec - 64.0 62.62 -2.16 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_combine_1 sparkstructuredstreaming_runtime_sec - 554.59 557.39 0.5 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_combine_4 dataflow_runtime_sec - 27.99 27.76 -0.8 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_combine_4 sparkstructuredstreaming_runtime_sec - 124.67 126.83 1.73 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_combine_5 dataflow_runtime_sec - 27.27 24.84 -8.93 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_combine_5 sparkstructuredstreaming_runtime_sec - 119.92 120.06 0.12 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_batch_gbk_1 dataflow_runtime_sec - 125.42 115.83 -7.64 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_1 dataflow_v2_java11_runtime_sec - 96.68 95.35 -1.38 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_1 dataflow_v2_java17_runtime_sec - 86.51 90.35 4.44 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_1 sparkstructuredstreaming_runtime_sec - 502.91 503.81 0.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_2 dataflow_runtime_sec - 33.68 32.33 -4.0 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_2 dataflow_v2_java11_runtime_sec - 42.27 40.93 -3.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_2 dataflow_v2_java17_runtime_sec - 39.31 40.94 4.12 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_2 sparkstructuredstreaming_runtime_sec - 97.8 107.11 9.52 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_3 dataflow_runtime_sec - 11.22 10.07 -10.3 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_3 dataflow_v2_java11_runtime_sec - 14.44 17.31 19.88 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_3 dataflow_v2_java17_runtime_sec - 15.28 15.39 0.75 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_3 sparkstructuredstreaming_runtime_sec - 8.2 8.74 6.56 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_4 dataflow_runtime_sec - 18.85 19.23 2.05 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_4 dataflow_v2_java11_runtime_sec - 22.58 22.18 -1.78 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_4 dataflow_v2_java17_runtime_sec - 23.02 24.62 6.92 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_4 sparkstructuredstreaming_runtime_sec - 97.45 101.76 4.43 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_5 dataflow_runtime_sec - 18.5 17.83 -3.64 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_5 dataflow_v2_java11_runtime_sec - 17.53 17.45 -0.49 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_5 dataflow_v2_java17_runtime_sec - 19.43 20.54 5.68 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_5 sparkstructuredstreaming_runtime_sec - 100.52 100.52 -0.0 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_6 dataflow_runtime_sec - 37.2 34.88 -6.26 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_6 dataflow_v2_java11_runtime_sec - 26.4 26.36 -0.12 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_6 dataflow_v2_java17_runtime_sec - 31.33 26.3 -16.06 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_6 sparkstructuredstreaming_runtime_sec - 58.34 60.86 4.32 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_runtime_sec - 49.25 44.03 -10.62 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_v2_java11_runtime_sec - 42.23 34.58 -18.11 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 dataflow_v2_java17_runtime_sec - 35.86 39.61 10.45 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_gbk_7 sparkstructuredstreaming_runtime_sec - 84.12 92.67 10.17 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_batch_pardo_1 dataflow_runtime_sec - 20.04 18.14 -9.49 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_1 dataflow_v2_java11_runtime_sec - 13.25 16.08 21.34 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_1 dataflow_v2_java17_runtime_sec - 15.09 16.7 10.66 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_1 sparkstructuredstreaming_runtime_sec - 56.17 49.55 -11.8 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_2 dataflow_runtime_sec - 186.08 202.59 8.87 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_2 dataflow_v2_java11_runtime_sec - 63.83 71.69 12.31 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_2 dataflow_v2_java17_runtime_sec - 66.38 67.13 1.13 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_runtime_sec - 13.8 12.29 -10.98 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_v2_java11_runtime_sec - 9.09 15.75 73.32 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_3 dataflow_v2_java17_runtime_sec - 12.12 14.5 19.67 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_4 dataflow_runtime_sec - 24.8 22.52 -9.18 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_4 dataflow_v2_java11_runtime_sec - 26.58 24.46 -7.98 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_batch_pardo_4 dataflow_v2_java17_runtime_sec - 25.41 24.93 -1.9 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_cogbk_1 dataflow_runtime_sec - 927.84 858.74 -7.45 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_1 dataflow_v2_java11_runtime_sec - 3095.03 2628.35 -15.08 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_1 dataflow_v2_java17_runtime_sec - 2589.29 2595.64 0.25 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_2 dataflow_runtime_sec - 907.41 837.58 -7.7 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_2 dataflow_v2_java11_runtime_sec - 3256.56 2969.53 -8.81 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_2 dataflow_v2_java17_runtime_sec - 2938.01 3090.66 5.2 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_3 dataflow_runtime_sec - 118.04 122.42 3.71 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_3 dataflow_v2_java11_runtime_sec - 322.79 326.12 1.03 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_3 dataflow_v2_java17_runtime_sec - 319.04 325.44 2.01 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_4 dataflow_runtime_sec - 125.95 122.06 -3.09 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_4 dataflow_v2_java11_runtime_sec - 320.92 326.47 1.73 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_cogbk_4 dataflow_v2_java17_runtime_sec - 331.9 322.66 -2.78 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> java_streaming_combine_1 dataflow_runtime_sec - 509.03 511.79 0.54 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_streaming_combine_4 dataflow_runtime_sec - 318.94 322.22 1.03 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_streaming_combine_5 dataflow_runtime_sec - 307.67 309.5 0.6 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> java_streaming_gbk_1 dataflow_runtime_sec - 670.68 646.17 -3.65 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_1 dataflow_v2_java11_runtime_sec - 875.0 920.0 5.14 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_1 dataflow_v2_java17_runtime_sec - 1003.04 897.89 -10.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_2 dataflow_runtime_sec - 803.33 798.42 -0.61 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_2 dataflow_v2_java11_runtime_sec - 4656.02 2798.25 -39.9 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_2 dataflow_v2_java17_runtime_sec - 3158.86 3166.16 0.23 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_3 dataflow_runtime_sec - 23.62 24.92 5.49 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_3 dataflow_v2_java11_runtime_sec - 53.16 59.27 11.48 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_3 dataflow_v2_java17_runtime_sec - 69.2 53.75 -22.32 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_4 dataflow_runtime_sec - 289.5 286.49 -1.04 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_4 dataflow_v2_java11_runtime_sec - 641.36 649.91 1.33 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_4 dataflow_v2_java17_runtime_sec - 665.68 685.61 2.99 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_5 dataflow_runtime_sec - 293.31 280.95 -4.21 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_5 dataflow_v2_java11_runtime_sec - 602.18 632.83 5.09 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_5 dataflow_v2_java17_runtime_sec - 677.63 555.04 -18.09 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_6 dataflow_runtime_sec - 73.86 76.09 3.02 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_6 dataflow_v2_java11_runtime_sec - 160.18 145.57 -9.12 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_6 dataflow_v2_java17_runtime_sec - 194.24 178.72 -7.99 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_7 dataflow_runtime_sec - 137.68 137.62 -0.04 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_7 dataflow_v2_java11_runtime_sec - 207.62 226.19 8.94 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_gbk_7 dataflow_v2_java17_runtime_sec - 223.14 202.22 -9.37 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> java_streaming_pardo_1 dataflow_runtime_sec - 18.79 18.93 0.75 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_1 dataflow_v2_java11_runtime_sec - 28.86 23.28 -19.35 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_1 dataflow_v2_java17_runtime_sec - 25.27 24.23 -4.08 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_2 dataflow_runtime_sec - 223.51 226.23 1.22 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_2 dataflow_v2_java11_runtime_sec - 114.31 111.55 -2.41 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_2 dataflow_v2_java17_runtime_sec - 105.51 97.51 -7.58 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_3 dataflow_runtime_sec - 12.55 12.71 1.25 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_3 dataflow_v2_java11_runtime_sec - 21.3 23.56 10.62 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_3 dataflow_v2_java17_runtime_sec - 24.03 24.17 0.61 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_4 dataflow_runtime_sec - 32.98 33.47 1.49 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_4 dataflow_v2_java11_runtime_sec - 36.78 35.46 -3.61 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> java_streaming_pardo_4 dataflow_v2_java17_runtime_sec - 37.98 33.86 -10.85 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> jdbcioit_results read_time - 24.89 24.82 -0.28 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> jdbcioit_results write_time - 38.16 36.52 -4.3 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results read_time - 777.55 0.0 -100.0 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results write_time - 212.65 14.37 -93.24 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results_runner_v2 read_time - 1.87 2.36 26.33 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results_runner_v2 write_time - 5.89 5.91 0.4 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results_sdf_wrapper read_time - 3.04 3.26 7.08 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> kafkaioit_results_sdf_wrapper write_time - 6.25 3.97 -36.43 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> many_files_textioit_hdfs_results read_time - 17.29 17.48 1.13 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> many_files_textioit_hdfs_results write_time - 61.16 56.82 -7.09 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> many_files_textioit_results read_time - 24.9 24.23 -2.67 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> many_files_textioit_results write_time - 61.41 58.16 -5.29 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> mongodbioit_results read_time - 30.18 30.83 2.15 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> mongodbioit_results write_time - 59.02 55.45 -6.06 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> parquetioit_hdfs_results read_time - 88.2 88.97 0.88 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> parquetioit_hdfs_results write_time - 147.1 142.12 -3.39 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> parquetioit_results read_time - 87.88 89.9 2.29 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> parquetioit_results write_time - 145.06 139.23 -4.02 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> python_batch_cogbk_1 python_dataflow_batch_cogbk_1_runtime - 525.29 453.8 -13.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_cogbk_2 python_dataflow_batch_cogbk_2_runtime - 287.43 317.2 10.36 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_cogbk_3 python_dataflow_batch_cogbk_3_runtime - 282.71 253.8 -10.23 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_cogbk_4 python_dataflow_batch_cogbk_4_runtime - 291.71 278.4 -4.56 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_batch_combine_1 python_dataflow_batch_combine_1_runtime - 841.29 875.6 4.08 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> python_batch_combine_4 python_dataflow_batch_combine_4_runtime - 61.86 53.0 -14.32 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> python_batch_combine_5 python_dataflow_batch_combine_5_runtime - 46.0 44.4 -3.48 [1] <http://metrics.beam.apache.org/d/WNzYt13Zk>
> python_batch_gbk_1 python_dataflow_batch_gbk_1_runtime - 1877.29 1915.8 2.05 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_2 python_dataflow_batch_gbk_2_runtime - 373.14 367.8 -1.43 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_3 python_dataflow_batch_gbk_3_runtime - 27.43 32.6 18.85 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_4 python_dataflow_batch_gbk_4_runtime - 124.43 124.2 -0.18 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_5 python_dataflow_batch_gbk_5_runtime - 127.0 129.4 1.89 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_6 python_dataflow_batch_gbk_6_runtime - 344.86 352.4 2.19 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_gbk_7 python_dataflow_batch_gbk_7_runtime - 353.14 346.0 -2.02 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_batch_pardo_1 python_dataflow_batch_pardo_1_runtime - 180.14 176.8 -1.86 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_batch_pardo_2 python_dataflow_batch_pardo_2_runtime - 1635.57 1661.4 1.58 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_batch_pardo_3 python_dataflow_batch_pardo_3_runtime - 141.86 150.6 6.16 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_batch_pardo_4 python_dataflow_batch_pardo_4_runtime - 326.71 345.2 5.66 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_batch_sideinput_1 python_dataflow_batch_sideinput_1_runtime - 47.86 46.4 -3.04 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_2 python_dataflow_batch_sideinput_2_runtime - 50.14 45.0 -10.26 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_3 python_dataflow_batch_sideinput_3_runtime - 5.71 5.4 -5.5 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_4 python_dataflow_batch_sideinput_4_runtime - 471.29 436.4 -7.4 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_5 python_dataflow_batch_sideinput_5_runtime - 47.29 44.4 -6.1 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_6 python_dataflow_batch_sideinput_6_runtime - 45.29 38.0 -16.09 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_7 python_dataflow_batch_sideinput_7_runtime - 5566.29 5588.2 0.39 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_batch_sideinput_8 python_dataflow_batch_sideinput_8_runtime - 5592.14 5562.8 -0.52 [1] <http://metrics.beam.apache.org/d/-E9aGlFGk>
> python_bqio_read_10GB_results bqio_read_10GB_results_runtime - 118.71 111.6 -5.99 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
> python_bqio_write_10GB_results bqio_write_10GB_results_runtime - 247.86 210.2 -15.19 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
> python_direct_microbenchmarks fn_api_runner_microbenchmark_runtime_sec - 94.83 94.11 -0.76
> python_direct_microbenchmarks teststream_microbenchmark_runtime_sec - 344.58 337.86 -1.95
> python_spanner_write_2GB_results pyspanner_write_2GB_results_runtime - 1562.57 1599.2 2.34 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
> python_spannerio_read_2GB_results pyspanner_read_2GB_results_runtime - 26.43 23.8 -9.95 [1] <http://metrics.beam.apache.org/d/gP7vMPqZz>
> python_streaming_cogbk_1 python_dataflow_streaming_cogbk_1_runtime - 628.29 659.8 5.02 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_streaming_cogbk_2 python_dataflow_streaming_cogbk_2_runtime - 425.0 422.4 -0.61 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_streaming_cogbk_3 python_dataflow_streaming_cogbk_3_runtime - 506.43 491.2 -3.01 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_streaming_cogbk_4 python_dataflow_streaming_cogbk_4_runtime - 424.71 427.0 0.54 [1] <http://metrics.beam.apache.org/d/fK0U4JqWz>
> python_streaming_gbk_3 python_dataflow_streaming_gbk_3_runtime - 90.86 78.5 -13.6 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_streaming_gbk_6 python_dataflow_streaming_gbk_6_runtime - 523.57 546.0 4.28 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_streaming_gbk_7 python_dataflow_streaming_gbk_7_runtime - 506.29 553.5 9.33 [1] <http://metrics.beam.apache.org/d/UYZ-oJ3Zk>
> python_streaming_pardo_1 python_dataflow_streaming_pardo_1_runtime - 253.57 235.5 -7.13 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_streaming_pardo_2 python_dataflow_streaming_pardo_2_runtime - 2941.57 3096.5 5.27 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_streaming_pardo_3 python_dataflow_streaming_pardo_3_runtime - 204.0 200.0 -1.96 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> python_streaming_pardo_4 python_dataflow_streaming_pardo_4_runtime - 538.57 602.0 11.78 [1] <http://metrics.beam.apache.org/d/MOi-kf3Zk>
> textioit_hdfs_results read_time - 19.88 20.73 4.27 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> textioit_hdfs_results write_time - 32.34 31.18 -3.59 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> textioit_results read_time - 20.52 21.58 5.16 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> textioit_results write_time - 32.12 32.81 2.14 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> tfrecordioit_results read_time - 15.43 16.26 5.35 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> tfrecordioit_results write_time - 20.79 21.14 1.68 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> wordcount_py37_results runtime - 811.08 824.06 1.6 [1] <http://metrics.beam.apache.org/d/1cnwVDkGk>
> xmlioit_hdfs_results read_time - 21.4 22.77 6.44 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> xmlioit_hdfs_results write_time - 24.67 25.12 1.79 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> xmlioit_results read_time - 23.48 23.36 -0.53 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> xmlioit_results write_time - 24.44 26.24 7.36 [1] <http://metrics.beam.apache.org/d/bnlHKP3Wz>
> nexmark_0_batch RuntimeMs DataflowRunner 27146.02 27212.08 0.24 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_batch RuntimeMs DirectRunner 4782.36 4783.0 0.01 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_batch RuntimeMs FlinkRunner 1155.07 1111.86 -3.74 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_0_batch RuntimeMs SparkRunner 1253.64 1255.18 0.12 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_batch RuntimeMs SparkStructuredStreamingRunner 1546.91 1554.18 0.47 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_0_sql_batch RuntimeMs DataflowRunner 35274.21 35610.5 0.95 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_0_sql_batch RuntimeMs DirectRunner 5570.26 5655.59 1.53 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_sql_batch RuntimeMs SparkRunner 3205.2 3141.48 -1.99 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_0_sql_batch RuntimeMs SparkStructuredStreamingRunner 2939.46 2926.11 -0.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_0_sql_streaming RuntimeMs DataflowRunner 24026.89 23121.55 -3.77 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_sql_streaming RuntimeMs DirectRunner 15501.0 15555.91 0.35 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_sql_streaming RuntimeMs FlinkRunner 1637.46 1628.64 -0.54 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_0_streaming RuntimeMs DataflowRunner 39291.92 37816.14 -3.76 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_streaming RuntimeMs DirectRunner 11811.79 11938.45 1.07 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_streaming RuntimeMs FlinkRunner 1432.14 1422.36 -0.68 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_0_zetasql_batch RuntimeMs DataflowRunner 114915.96 112238.45 -2.33 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_0_zetasql_batch RuntimeMs DirectRunner 6757.07 6834.27 1.14 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_0_zetasql_streaming RuntimeMs DataflowRunner 67751.04 66627.91 -1.66 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_zetasql_streaming RuntimeMs DirectRunner 17105.75 16996.45 -0.64 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_0_zetasql_streaming RuntimeMs FlinkRunner 3248.54 3261.41 0.4 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_10_batch RuntimeMs DataflowRunner 48690.3 49306.87 1.27 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_10_batch RuntimeMs DirectRunner 10463.32 10465.09 0.02 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_10_batch RuntimeMs FlinkRunner 777.21 714.64 -8.05 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_10_batch RuntimeMs SparkRunner 726.5 729.61 0.43 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_10_batch RuntimeMs SparkStructuredStreamingRunner 1126.52 1211.0 7.5 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_10_streaming RuntimeMs DataflowRunner 105776.61 102899.66 -2.72 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_10_streaming RuntimeMs DirectRunner 43926.21 44091.14 0.38 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_10_streaming RuntimeMs FlinkRunner 1535.29 1488.14 -3.07 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_11_batch RuntimeMs DataflowRunner 39589.51 61325.57 54.9 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_11_batch RuntimeMs DirectRunner 3757.79 3770.55 0.34 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_11_batch RuntimeMs FlinkRunner 297.43 313.64 5.45 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_11_batch RuntimeMs SparkRunner 1402.09 1399.52 -0.18 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_11_batch RuntimeMs SparkStructuredStreamingRunner 1135.95 1015.14 -10.64 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_11_streaming RuntimeMs DataflowRunner 182507.05 181193.81 -0.72 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_11_streaming RuntimeMs DirectRunner 10696.0 10739.0 0.4 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_11_streaming RuntimeMs FlinkRunner 1351.79 1319.05 -2.42 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_12_batch RuntimeMs DataflowRunner 19630.69 19666.75 0.18 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_12_batch RuntimeMs DirectRunner 2597.07 2626.36 1.13 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_12_batch RuntimeMs FlinkRunner 180.29 173.05 -4.02 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_12_batch RuntimeMs SparkRunner 924.52 929.25 0.51 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_12_batch RuntimeMs SparkStructuredStreamingRunner 827.98 899.98 8.7 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_12_streaming RuntimeMs DataflowRunner 96217.51 94780.91 -1.49 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_12_streaming RuntimeMs DirectRunner 9335.68 9409.5 0.79 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_12_streaming RuntimeMs FlinkRunner 621.5 620.73 -0.12 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_batch RuntimeMs DataflowRunner 33870.18 33840.36 -0.09 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_batch RuntimeMs DirectRunner 3225.36 3231.86 0.2 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_13_batch RuntimeMs FlinkRunner 392.68 404.95 3.13 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_13_batch RuntimeMs SparkRunner 355.27 357.02 0.49 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_batch RuntimeMs SparkStructuredStreamingRunner 547.32 556.57 1.69 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_13_sql_batch RuntimeMs DataflowRunner 62806.75 61308.68 -2.39 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_13_sql_batch RuntimeMs DirectRunner 9065.15 9052.5 -0.14 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_sql_batch RuntimeMs SparkRunner 3473.45 3430.11 -1.25 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_sql_batch RuntimeMs SparkStructuredStreamingRunner 3113.52 3060.59 -1.7 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_sql_streaming RuntimeMs DataflowRunner 42997.25 34577.23 -19.58 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_sql_streaming RuntimeMs DirectRunner 18918.29 18631.09 -1.52 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_sql_streaming RuntimeMs FlinkRunner 28617.96 29781.77 4.07 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_streaming RuntimeMs DataflowRunner 28812.61 24063.0 -16.48 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_streaming RuntimeMs DirectRunner 9996.14 9949.18 -0.47 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_streaming RuntimeMs FlinkRunner 27765.36 28396.95 2.27 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_zetasql_batch RuntimeMs DataflowRunner 262683.96 259338.23 -1.27 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_13_zetasql_batch RuntimeMs DirectRunner 10046.04 10059.45 0.13 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_13_zetasql_streaming RuntimeMs DataflowRunner 116689.18 109084.86 -6.52 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_zetasql_streaming RuntimeMs DirectRunner 21036.07 21342.05 1.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_13_zetasql_streaming RuntimeMs FlinkRunner 34089.0 34659.64 1.67 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_14_batch RuntimeMs DataflowRunner 62350.36 61308.36 -1.67 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_14_batch RuntimeMs DirectRunner 4153.39 4150.05 -0.08 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_14_batch RuntimeMs FlinkRunner 666.86 650.5 -2.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_14_batch RuntimeMs SparkRunner 895.64 945.5 5.57 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_14_batch RuntimeMs SparkStructuredStreamingRunner 933.43 941.27 0.84 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_14_streaming RuntimeMs DataflowRunner 121083.79 126024.55 4.08 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_14_streaming RuntimeMs DirectRunner 12338.79 12202.36 -1.11 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_14_streaming RuntimeMs FlinkRunner 1783.46 1762.27 -1.19 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_15_batch RuntimeMs DataflowRunner 45464.97 45823.83 0.79 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_15_batch RuntimeMs DirectRunner 15421.04 15413.5 -0.05 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_15_batch RuntimeMs FlinkRunner 646.39 579.05 -10.42 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_15_batch RuntimeMs SparkRunner 1643.96 1675.16 1.9 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_15_batch RuntimeMs SparkStructuredStreamingRunner 1451.93 1442.11 -0.68 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_16_batch RuntimeMs DataflowRunner 51963.45 51419.82 -1.05 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_16_batch RuntimeMs DirectRunner 85652.86 84922.68 -0.85 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_16_batch RuntimeMs FlinkRunner 214.46 256.23 19.47 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_16_batch RuntimeMs SparkRunner 1484.05 1386.3 -6.59 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_16_batch RuntimeMs SparkStructuredStreamingRunner 1249.75 1199.98 -3.98 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_16_streaming RuntimeMs DataflowRunner 89716.22 95450.93 6.39 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_16_streaming RuntimeMs DirectRunner 124442.11 129482.0 4.05 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_16_streaming RuntimeMs FlinkRunner 423.43 431.27 1.85 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_batch RuntimeMs DataflowRunner 21877.32 20846.38 -4.71 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_batch RuntimeMs DirectRunner 3502.18 3394.41 -3.08 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_1_batch RuntimeMs FlinkRunner 315.89 307.91 -2.53 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_1_batch RuntimeMs SparkRunner 440.91 442.09 0.27 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_1_batch RuntimeMs SparkStructuredStreamingRunner 591.54 592.73 0.2 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_1_sql_batch RuntimeMs DataflowRunner 32344.21 32125.86 -0.68 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_1_sql_batch RuntimeMs DirectRunner 3546.3 3726.64 5.09 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_sql_batch RuntimeMs SparkRunner 2430.38 2390.05 -1.66 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_1_sql_batch RuntimeMs SparkStructuredStreamingRunner 1572.41 1666.39 5.98 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_1_sql_streaming RuntimeMs DataflowRunner 20870.43 21097.45 1.09 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_1_sql_streaming RuntimeMs DirectRunner 11466.04 11491.86 0.23 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_sql_streaming RuntimeMs FlinkRunner 667.54 637.77 -4.46 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_streaming RuntimeMs DataflowRunner 28937.91 28601.91 -1.16 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_streaming RuntimeMs DirectRunner 8206.14 8198.05 -0.1 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_1_streaming RuntimeMs FlinkRunner 648.86 667.5 2.87 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_batch RuntimeMs DataflowRunner 15263.96 15124.28 -0.92 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_2_batch RuntimeMs DirectRunner 1530.89 1605.64 4.88 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_2_batch RuntimeMs FlinkRunner 219.18 192.68 -12.09 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_batch RuntimeMs SparkRunner 311.59 310.82 -0.25 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_batch RuntimeMs SparkStructuredStreamingRunner 458.84 460.77 0.42 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_2_sql_batch RuntimeMs DataflowRunner 25789.39 25085.68 -2.73 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_2_sql_batch RuntimeMs DirectRunner 1543.37 1555.64 0.79 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_2_sql_batch RuntimeMs SparkRunner 744.41 744.02 -0.05 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_sql_batch RuntimeMs SparkStructuredStreamingRunner 1005.04 972.02 -3.28 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_2_sql_streaming RuntimeMs DataflowRunner 13841.29 13565.55 -1.99 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_sql_streaming RuntimeMs DirectRunner 6789.04 6801.5 0.18 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_sql_streaming RuntimeMs FlinkRunner 424.61 410.73 -3.27 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_streaming RuntimeMs DataflowRunner 19257.58 18899.86 -1.86 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_streaming RuntimeMs DirectRunner 5142.04 5144.05 0.04 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_streaming RuntimeMs FlinkRunner 270.54 275.0 1.65 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_zetasql_batch RuntimeMs DataflowRunner 71538.04 73956.5 3.38 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_zetasql_batch RuntimeMs DirectRunner 2908.36 2883.55 -0.85 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_zetasql_streaming RuntimeMs DataflowRunner 39600.89 40732.09 2.86 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_2_zetasql_streaming RuntimeMs DirectRunner 8218.75 8148.45 -0.86 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_2_zetasql_streaming RuntimeMs FlinkRunner 1202.64 1165.27 -3.11 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_batch RuntimeMs DataflowRunner 26748.14 27266.41 1.94 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_batch RuntimeMs DirectRunner 3224.14 3243.5 0.6 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_batch RuntimeMs FlinkRunner 1105.75 1059.41 -4.19 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_3_batch RuntimeMs SparkRunner 2086.2 2081.61 -0.22 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_sql_batch RuntimeMs DataflowRunner 29239.46 29789.05 1.88 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_3_sql_batch RuntimeMs DirectRunner 3365.15 3329.5 -1.06 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_3_sql_batch RuntimeMs SparkRunner 3503.88 3480.3 -0.67 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_3_sql_batch RuntimeMs SparkStructuredStreamingRunner 3032.75 2996.09 -1.21 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_3_sql_streaming RuntimeMs DataflowRunner 33163.79 33241.18 0.23 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_sql_streaming RuntimeMs DirectRunner 8460.71 8474.64 0.16 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_sql_streaming RuntimeMs FlinkRunner 970.11 960.36 -1.0 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_streaming RuntimeMs DataflowRunner 43269.14 43554.49 0.66 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_streaming RuntimeMs DirectRunner 7623.46 7534.73 -1.16 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_streaming RuntimeMs FlinkRunner 469.46 456.77 -2.7 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_zetasql_batch RuntimeMs DataflowRunner 54330.46 54539.23 0.38 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_3_zetasql_batch RuntimeMs DirectRunner 3749.36 3745.59 -0.1 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_zetasql_streaming RuntimeMs DataflowRunner 46361.32 46243.59 -0.25 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_3_zetasql_streaming RuntimeMs DirectRunner 8915.93 8861.45 -0.61 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_3_zetasql_streaming RuntimeMs FlinkRunner 1191.43 1197.64 0.52 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_4_batch RuntimeMs DataflowRunner 33212.25 33292.5 0.24 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_4_batch RuntimeMs DirectRunner 2416.46 2409.05 -0.31 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_4_batch RuntimeMs FlinkRunner 861.68 851.14 -1.22 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_4_batch RuntimeMs SparkRunner 1001.52 998.98 -0.25 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_4_batch RuntimeMs SparkStructuredStreamingRunner 2243.59 2233.52 -0.45 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_4_streaming RuntimeMs DataflowRunner 49175.82 49133.95 -0.09 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_4_streaming RuntimeMs DirectRunner 2827.5 2862.05 1.22 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_4_streaming RuntimeMs FlinkRunner 975.32 1021.45 4.73 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_5_batch RuntimeMs DataflowRunner 35151.96 34758.3 -1.12 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_5_batch RuntimeMs DirectRunner 10582.36 10621.95 0.37 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_5_batch RuntimeMs FlinkRunner 775.96 769.41 -0.84 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_5_batch RuntimeMs SparkRunner 1515.27 1535.14 1.31 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_5_batch RuntimeMs SparkStructuredStreamingRunner 1719.73 1732.2 0.73 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_5_streaming RuntimeMs DataflowRunner 214447.03 209021.26 -2.53 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_5_streaming RuntimeMs DirectRunner 20728.25 21227.32 2.41 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_5_streaming RuntimeMs FlinkRunner 1609.86 1637.59 1.72 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_6_batch RuntimeMs DataflowRunner 31298.14 31001.5 -0.95 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_6_batch RuntimeMs DirectRunner 1886.82 1851.91 -1.85 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_6_batch RuntimeMs FlinkRunner 536.79 519.91 -3.14 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_6_batch RuntimeMs SparkRunner 710.09 706.14 -0.56 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_6_batch RuntimeMs SparkStructuredStreamingRunner 907.57 948.52 4.51 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_6_streaming RuntimeMs DataflowRunner 38287.43 41809.0 9.2 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_6_streaming RuntimeMs DirectRunner 2135.57 2098.95 -1.71 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_6_streaming RuntimeMs FlinkRunner 601.57 592.18 -1.56 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_batch RuntimeMs DataflowRunner 54975.39 56478.73 2.73 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_7_batch RuntimeMs DirectRunner 18420.82 18547.23 0.69 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_7_batch RuntimeMs FlinkRunner 665.07 677.59 1.88 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_7_batch RuntimeMs SparkRunner 1440.04 1426.75 -0.92 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_7_batch RuntimeMs SparkStructuredStreamingRunner 2214.61 2196.27 -0.83 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_7_sql_batch RuntimeMs DataflowRunner 78570.29 78734.55 0.21 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_7_sql_batch RuntimeMs DirectRunner 326788.15 364568.64 11.56 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_sql_batch RuntimeMs SparkRunner 7212.82 7068.95 -1.99 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_sql_batch RuntimeMs SparkStructuredStreamingRunner 5996.48 5910.11 -1.44 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_7_sql_streaming RuntimeMs DataflowRunner 1003130.36 1027200.23 2.4 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_7_sql_streaming RuntimeMs DirectRunner 1076402.96 1215625.41 12.93 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_sql_streaming RuntimeMs FlinkRunner 3486.11 3459.86 -0.75 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_7_streaming RuntimeMs DataflowRunner 120673.19 122350.28 1.39 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_7_streaming RuntimeMs DirectRunner 850907.18 848503.82 -0.28 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_7_streaming RuntimeMs FlinkRunner 13410.82 13769.05 2.67 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_8_batch RuntimeMs DataflowRunner 21572.36 21636.01 0.3 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_8_batch RuntimeMs DirectRunner 2258.14 2383.27 5.54 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/tkqc0AdGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_8_batch RuntimeMs FlinkRunner 473.39 478.09 0.99 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_8_batch RuntimeMs SparkRunner 1451.93 1434.59 -1.19 [1] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_8_batch RuntimeMs SparkStructuredStreamingRunner 1185.75 1128.57 -4.82 [1] <http://metrics.beam.apache.org/d/tkqc0AdGk> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_8_streaming RuntimeMs DataflowRunner 59645.41 61147.95 2.52 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_8_streaming RuntimeMs DirectRunner 6981.75 7187.59 2.95 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_8_streaming RuntimeMs FlinkRunner 479.46 478.59 -0.18 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz>
> nexmark_9_batch RuntimeMs DataflowRunner 29188.29 30279.18 3.74 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz> [3] <http://metrics.beam.apache.org/d/f0Q6T0dGk>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_9_batch RuntimeMs DirectRunner 1454.29 1440.0 -0.98 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/ahudA_zGz> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/f0Q6T0dGk>
> nexmark_9_batch RuntimeMs FlinkRunner 290.18 325.45 12.16 [1] <http://metrics.beam.apache.org/d/8INnSY9Mz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/ahudA_zGz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_9_batch RuntimeMs SparkRunner 541.73 544.66 0.54 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/tkqc0AdGk>[4] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_9_batch RuntimeMs SparkStructuredStreamingRunner 708.32 716.98 1.22 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/f0Q6T0dGk> [3] <http://metrics.beam.apache.org/d/8INnSY9Mz>[4] <http://metrics.beam.apache.org/d/tkqc0AdGk>
> nexmark_9_streaming RuntimeMs DataflowRunner 31862.82 31803.18 -0.19 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_9_streaming RuntimeMs DirectRunner 1758.86 1774.09 0.87 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>
> nexmark_9_streaming RuntimeMs FlinkRunner 430.46 487.23 13.19 [1] <http://metrics.beam.apache.org/d/ahudA_zGz> [2] <http://metrics.beam.apache.org/d/8INnSY9Mz>