[jira] [Comment Edited] (FLINK-23352) Collect Sink Operator Factory does not support to configure max bytes per batch

["Caizhi Weng (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-23352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17379026#comment-17379026 ] 

Caizhi Weng edited comment on FLINK-23352 at 7/12/21, 7:59 AM:
---------------------------------------------------------------

[~jark] This issue is actually from a user request. The user is collecting rows with super large size (one row is bigger than 2MB). Current implementation of collect sink has to transmit a complete row in one batch, so if the row is too large to fit into a batch collect sink will refuse to work.


was (Author: tsreaper):
[~jark] This issue is actually from a user request. The user is collecting rows with super large size (one row is bigger than 2MB). Currently implementation of collect sink has to transmit a complete row in one batch, so if the row is too large to fit into a batch collect sink will refuse to work.

> Collect Sink Operator Factory does not support to configure max bytes per batch
> -------------------------------------------------------------------------------
>
>                 Key: FLINK-23352
>                 URL: https://issues.apache.org/jira/browse/FLINK-23352
>             Project: Flink
>          Issue Type: Improvement
>          Components: API / DataStream, Table SQL / Ecosystem
>    Affects Versions: 1.13.1, 1.12.4
>            Reporter: Yun Tang
>            Priority: Major
>
> Current {{CollectSinkOperatorFactory}} only supports max bytes per batch as 2MB. However, this configuration might not be enough for production users and we should provide ability to make this configurable. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)