You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Anton Kalashnikov (Jira)" <ji...@apache.org> on 2021/09/08 12:32:00 UTC
[jira] [Updated] (FLINK-23974) Low throughput for buffer debloat in
case of different load profile
[ https://issues.apache.org/jira/browse/FLINK-23974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anton Kalashnikov updated FLINK-23974:
--------------------------------------
Description:
According to task https://issues.apache.org/jira/browse/FLINK-23456, we have some performance drop when the buffer debloat is enabled:
* for static load with throttling in source - 1-3%
* for sinus shape load with throttling in source - 2-3%
* for erratic load with throttling in source - 5-6%
It needs to investigate the reason for that and try to improve if it is possible.
It makes sense to write the microbenchmarks for these scenarios to reproduce the problem locally.
Problem assumption:
Highly likely the problem is the speed of reaction on increasing load. During the investigation, it needs to pay attention to the calculation throughput for different load profiles. It is possible that when the load is low we are decreasing the buffer size to a small value(maybe minimum value) and when the load gets back to high we need several iterations of the throughput calculation to reach the appropriate buffer size.
was:
According to task https://issues.apache.org/jira/browse/FLINK-23456, we have some performance drop when the buffer debloat is enabled:
* for static load with throttling in source - 1-3%
* for sinus shape load with throttling in source - 2-3%
* for erratic load with throttling in source - 5-6%
It needs to investigate the reason for that and try to improve if it is possible.
> Low throughput for buffer debloat in case of different load profile
> -------------------------------------------------------------------
>
> Key: FLINK-23974
> URL: https://issues.apache.org/jira/browse/FLINK-23974
> Project: Flink
> Issue Type: Sub-task
> Reporter: Anton Kalashnikov
> Priority: Major
>
> According to task https://issues.apache.org/jira/browse/FLINK-23456, we have some performance drop when the buffer debloat is enabled:
> * for static load with throttling in source - 1-3%
> * for sinus shape load with throttling in source - 2-3%
> * for erratic load with throttling in source - 5-6%
> It needs to investigate the reason for that and try to improve if it is possible.
> It makes sense to write the microbenchmarks for these scenarios to reproduce the problem locally.
> Problem assumption:
> Highly likely the problem is the speed of reaction on increasing load. During the investigation, it needs to pay attention to the calculation throughput for different load profiles. It is possible that when the load is low we are decreasing the buffer size to a small value(maybe minimum value) and when the load gets back to high we need several iterations of the throughput calculation to reach the appropriate buffer size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)