You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Hangxiang Yu (Jira)" <ji...@apache.org> on 2023/01/12 03:08:00 UTC
[jira] [Comment Edited] (FLINK-30624) Performance regression in stateBackends.FS / FS_ASYNC / MEMORY on 05.01.2023
[ https://issues.apache.org/jira/browse/FLINK-30624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17675791#comment-17675791 ]
Hangxiang Yu edited comment on FLINK-30624 at 1/12/23 3:07 AM:
---------------------------------------------------------------
I guess the regression is caused by FLINK-30544.
In theory, FLINK-30544 trade o(n) space which is located at heap for o(logn) time.
It will reduce heap spaces StateBackend could use so that make heap-related StateBackend like FS/MEMORY regress.
I also submitted benchmark requests to verify this:
* [179|http://codespeed.dak8s.net:8080/job/flink-benchmark-request/179/] – before FLINK-30544:
** stateBackends.MEMORY: 4177.022 (+-53.816)
** stateBackends.FS: 4153.435 (+-66.577)
** stateBackends.FS_ASYNC: 4139.056 (+-40.393)
* [180|http://codespeed.dak8s.net:8080/job/flink-benchmark-request/180/] - after FLINK-30544:
** stateBackends.MEMORY: 3936.158 (+-65.696)
** stateBackends.FS: 3963.934 (+-64.942)
** stateBackends.FS_ASYNC: 3930.955 (+-68.809)
So maybe [~wanglijie] could help to take a look ?
was (Author: masteryhx):
I guess the regression is caused by FLINK-30544.
In theory, FLINK-30544 trade o(n) space which is located at heap for o(logn) time.
It will reduce heap spaces StateBackend could use so that make heap-related StateBackend like FS/MEMORY regress.
I also submitted benchmark requests to verify this:
* [179|http://codespeed.dak8s.net:8080/job/flink-benchmark-request/179/] – before FLINK-30544:
** stateBackends.MEMORY: 4177.022 (+-53.816)
** stateBackends.FS: 4153.435 (+-66.577)
** stateBackends.FS_ASYNC: 4139.056 (+-40.393)
* [180|http://codespeed.dak8s.net:8080/job/flink-benchmark-request/180/] - after FLINK-30544:
** stateBackends.MEMORY: 3936.158 (+-65.696)
** stateBackends.FS: 3963.934 (+-64.942)
** stateBackends.FS_ASYNC: 3930.955 (+-68.809)
So maybe [~wanglijie] could help to take a look ?
> Performance regression in stateBackends.FS / FS_ASYNC / MEMORY on 05.01.2023
> ----------------------------------------------------------------------------
>
> Key: FLINK-30624
> URL: https://issues.apache.org/jira/browse/FLINK-30624
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Reporter: Martijn Visser
> Priority: Critical
>
> stateBackends.FS median=4147.01197 recent_median=3957.8419495
> stateBackends.FS_ASYNC median=4148.8160595 recent_median=3973.418166
> stateBackends.MEMORY median=4114.406091 recent_median=3935.8805775
> http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=stateBackends.FS&extr=on&quarts=on&equid=off&env=2&revs=200
> http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=stateBackends.FS_ASYNC&extr=on&quarts=on&equid=off&env=2&revs=200
> http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=stateBackends.MEMORY&extr=on&quarts=on&equid=off&env=2&revs=200
--
This message was sent by Atlassian Jira
(v8.20.10#820010)