[jira] [Commented] (FLINK-15171) Performance regression in serialisation benchmarks

["Roman Grebennikov (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16992587#comment-16992587 ] 

Roman Grebennikov commented on FLINK-15171:
-------------------------------------------

[~pnowojski] I've tried running these benchmarks on my hardware and got weird results:
{noformat}
1thread master SerializationFrameworkMiniBenchmarks.serializerTuple  thrpt  100  608.128 ± 8.701  ops/ms
1thread no-pr  SerializationFrameworkMiniBenchmarks.serializerTuple  thrpt  100  605.246 ± 9.584  ops/ms

4thread no-pr  SerializationFrameworkMiniBenchmarks.serializerTuple  thrpt  100  610.825 ± 10.945  ops/ms
4thread master SerializationFrameworkMiniBenchmarks.serializerTuple  thrpt  100  613.504 ± 8.557  ops/ms{noformat}
TLDR: no changes, as in original PR.

I may suspect that the difference may come out of different memory access patterns in the old and new version of serializers on different hardware: for example, my desktop having better memory throughput compared to the one used on dak8s.net.

Can you please describe a bit more the hardware on this Hetzner box used for benchmarking:
 * CPU model (cat /proc/cpuinfo)
 * memory details (cat /proc/meminfo)
 * hardware details (sudo dmidecode)

So I can try to reproduce the benchmark problem.

> Performance regression in serialisation benchmarks
> --------------------------------------------------
>
>                 Key: FLINK-15171
>                 URL: https://issues.apache.org/jira/browse/FLINK-15171
>             Project: Flink
>          Issue Type: Bug
>          Components: API / Type Serialization System, Benchmarks
>    Affects Versions: 1.10.0
>            Reporter: Piotr Nowojski
>            Priority: Blocker
>             Fix For: 1.10.0
>
>
> There is quite significant performance regression in serialisation benchmarks in the commit range 2ecf7ca..9320f34 (which includes FLINK-14346).
> http://codespeed.dak8s.net:8000/timeline/?ben=serializerTuple&env=2
> http://codespeed.dak8s.net:8000/timeline/?ben=serializerRow&env=2
> http://codespeed.dak8s.net:8000/timeline/?ben=serializerPojo&env=2
> it coincides with the performance improvement for heavy strings
> http://codespeed.dak8s.net:8000/timeline/?ben=serializerHeavyString&env=2
> it might be caused by some accidental change in the benchmarking code (changing parallelism in one benchmarks is carried on to the next one?) or in the code itself.
> CC [~rgrebennikov] [~AHeise]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)