You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Rui Fan (Jira)" <ji...@apache.org> on 2024/04/25 08:01:00 UTC
[jira] [Comment Edited] (FLINK-35215) The performance of serializerKryo and serializerKryoWithoutRegistration are regressed
[ https://issues.apache.org/jira/browse/FLINK-35215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840693#comment-17840693 ]
Rui Fan edited comment on FLINK-35215 at 4/25/24 8:00 AM:
----------------------------------------------------------
Here is the benchmark result for my test:
||Code||serializerKryo||serializerKryoWithoutRegistration||Comment||
|Case1: current master branch|167.163333333 ops/ms|135.03 ops/ms|Lower about 10% than reverts FLINK-34954|
|Case2: master branch reverts FLINK-34954 |190.636025667 ops/ms|147.184592333 ops/ms| |
|Case3: reverts FLINK-34954 and
with [https://github.com/apache/flink/pull/24717]|189.621157333 ops/ms|146.914767 ops/ms|Lower about 0.5% than reverts FLINK-34954 |
Note: I choose 3 samples for each cases.
h1. Case1: current master branch
data from benchmark result WebUI.
* serializerKryo (avg: 167.163333333 ops/ms)
** [http://flink-speed.xyz/timeline/#/?exe=6&ben=serializerKryo&extr=on&quarts=on&equid=off&env=3&revs=50]
** 164.68 ops/ms (Apr 22)
** 168.80 ops/ms (Apr 23)
** 168.01 ops/ms (Apr 24)
* serializerKryoWithoutRegistration (avg: 135.03 ops/ms)
** [http://flink-speed.xyz/timeline/#/?exe=12&ben=serializerKryoWithoutRegistration&extr=on&quarts=on&equid=off&env=3&revs=50]
** 134.93 ops/ms (Apr 22)
** 134.06 ops/ms (Apr 23)
** 136.10 ops/ms (Apr 24)
h1. Case2: master branch reverts FLINK-34954
I trigger it manually, the code branch is : [https://github.com/1996fanrui/flink/tree/revert-kyro-fix]
Note: the result of case2 and case3 isn't public, I paste the result from the Flink Community jenkins.
* serializerKryo (avg: 190.636025667 ops/ms)
** 194.929708 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/81/artifact/jmh-result.csv/*view*/]
** 186.023176 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/83/artifact/jmh-result.csv/*view*/]
** 190.955193 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/85/artifact/jmh-result.csv/*view*/]
* serializerKryoWithoutRegistration (avg: 147.184592333 ops/ms)
** 149.782267 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/81/artifact/jmh-result.csv/*view*/]
** 146.312711 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/83/artifact/jmh-result.csv/*view*/]
** 145.458799 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/85/artifact/jmh-result.csv/*view*/]
h1. Case3: reverts FLINK-34954 and with [https://github.com/apache/flink/pull/24717]
I trigger it manually, the code branch is : [https://github.com/1996fanrui/flink/tree/revert-kyro-fix-and-return-if-0]
* serializerKryo (avg: 189.621157333 ops/ms)
** 182.683889 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/82/artifact/jmh-result.csv/*view*/]
** 191.713318 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/84/artifact/jmh-result.csv/*view*/]
** 194.466265 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/86/artifact/jmh-result.csv/*view*/]
* serializerKryoWithoutRegistration (avg: 146.914767 ops/ms)
** 146.514048 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/82/artifact/jmh-result.csv/*view*/]
** 145.825161 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/84/artifact/jmh-result.csv/*view*/]
** 148.405092 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/86/artifact/jmh-result.csv/*view*/]
Note: I try to use other solution to fix the bug that FLINK-34954 mentioned, but I'm not sure whether the logic is right/suitable. If the solution is fine, the benchmark result is better than FLINK-34954.
Code change: [https://github.com/1996fanrui/flink/commit/ebce171a806a579225897a2d29e4cd28e9164c4f]
The change is pretty easy.
was (Author: fanrui):
Here is the benchmark result for my test:
||Code||serializerKryo||serializerKryoWithoutRegistration||Comment||
|Case1: current master branch|167.163333333 ops/ms|135.03 ops/ms|Lower about 10% than reverts FLINK-34954 |
|Case2: master branch reverts FLINK-34954 |190.636025667 ops/ms|147.184592333 ops/ms| |
|Case3: reverts FLINK-34954 and
with https://github.com/apache/flink/pull/24717|189.621157333 ops/ms|146.914767 ops/ms|Lower about 0.5% than reverts FLINK-34954 |
Note: I choose 3 samples for each cases.
h1. Case1: current master branch
data from benchmark result WebUI.
* serializerKryo (avg: 167.163333333 ops/ms)
** [http://flink-speed.xyz/timeline/#/?exe=6&ben=serializerKryo&extr=on&quarts=on&equid=off&env=3&revs=50]
** 164.68 ops/ms (Apr 22)
** 168.80 ops/ms (Apr 23)
** 168.01 ops/ms (Apr 24)
* serializerKryoWithoutRegistration (avg: 135.03 ops/ms)
** [http://flink-speed.xyz/timeline/#/?exe=12&ben=serializerKryoWithoutRegistration&extr=on&quarts=on&equid=off&env=3&revs=50]
** 134.93 ops/ms (Apr 22)
** 134.06 ops/ms (Apr 23)
** 136.10 ops/ms (Apr 24)
h1. Case2: master branch reverts FLINK-34954
I trigger it manually, the code branch is : [https://github.com/1996fanrui/flink/tree/revert-kyro-fix]
* serializerKryo (avg: 190.636025667 ops/ms)
** 194.929708 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/81/artifact/jmh-result.csv/*view*/]
** 186.023176 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/83/artifact/jmh-result.csv/*view*/]
** 190.955193 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/85/artifact/jmh-result.csv/*view*/]
* serializerKryoWithoutRegistration (avg: 147.184592333 ops/ms)
** 149.782267 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/81/artifact/jmh-result.csv/*view*/]
** 146.312711 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/83/artifact/jmh-result.csv/*view*/]
** 145.458799 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/85/artifact/jmh-result.csv/*view*/]
h1. Case3: reverts FLINK-34954 and with https://github.com/apache/flink/pull/24717
I trigger it manually, the code branch is : [https://github.com/1996fanrui/flink/tree/revert-kyro-fix-and-return-if-0]
* serializerKryo (avg: 189.621157333 ops/ms)
** 182.683889 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/82/artifact/jmh-result.csv/*view*/]
** 191.713318 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/84/artifact/jmh-result.csv/*view*/]
** 194.466265 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/86/artifact/jmh-result.csv/*view*/]
* serializerKryoWithoutRegistration (avg: 146.914767 ops/ms)
** 146.514048 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/82/artifact/jmh-result.csv/*view*/]
** 145.825161 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/84/artifact/jmh-result.csv/*view*/]
** 148.405092 ops/ms link: [http://jenkins.flink-speed.xyz/job/flink-benchmark-request/86/artifact/jmh-result.csv/*view*/]
Note: I try to use other solution to fix the bug that FLINK-34954 mentioned, but I'm not sure whether the logic is right/suitable. If the solution is fine, the benchmark result is better than FLINK-34954.
Code change: [https://github.com/1996fanrui/flink/commit/ebce171a806a579225897a2d29e4cd28e9164c4f]
The change is pretty easy.
> The performance of serializerKryo and serializerKryoWithoutRegistration are regressed
> -------------------------------------------------------------------------------------
>
> Key: FLINK-35215
> URL: https://issues.apache.org/jira/browse/FLINK-35215
> Project: Flink
> Issue Type: Bug
> Components: API / Type Serialization System
> Affects Versions: 1.20.0
> Reporter: Rui Fan
> Priority: Blocker
> Labels: pull-request-available
> Attachments: image-2024-04-25-14-57-55-231.png, image-2024-04-25-15-00-32-410.png
>
>
> The performance of serializerKryo and serializerKryoWithoutRegistration are regressed[1][2], I checked recent commits, and found FLINK-34954 changed related logic.
>
> [1] [http://flink-speed.xyz/timeline/#/?exe=1,6,12&ben=serializerKryo&extr=on&quarts=on&equid=off&env=3&revs=50]
> [2] http://flink-speed.xyz/timeline/#/?exe=1,6,12&ben=serializerKryoWithoutRegistration&extr=on&quarts=on&equid=off&env=3&revs=50
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)