You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yuxin Tan (Jira)" <ji...@apache.org> on 2023/08/22 07:54:00 UTC
[jira] [Updated] (FLINK-32870) Reading multiple small buffers by reading and slicing one large buffer for tiered storage
[ https://issues.apache.org/jira/browse/FLINK-32870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuxin Tan updated FLINK-32870:
------------------------------
Summary: Reading multiple small buffers by reading and slicing one large buffer for tiered storage (was: Merge multiple small buffers into one large buffer when loading data for tiered storage)
> Reading multiple small buffers by reading and slicing one large buffer for tiered storage
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-32870
> URL: https://issues.apache.org/jira/browse/FLINK-32870
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Network
> Affects Versions: 1.18.0
> Reporter: Yuxin Tan
> Assignee: Yuxin Tan
> Priority: Major
>
> Currently, when the file reader of tiered storage loads data from the disk file, it reads data in buffer granularity. Before compression, each buffer is 32K by default, after compression the size will become smaller (may less than 5K), which is pretty small for the network buffer and the file IO.
> We should merge the multiple small buffers into a larger one to decrease the buffer competition and the file IO, leading to better performance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)