You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Danny Cranmer (Jira)" <ji...@apache.org> on 2021/09/15 19:17:00 UTC
[jira] [Resolved] (FLINK-24234) [FLIP-171] Byte Based & Time Based
Flushing for AsyncSinkBase
[ https://issues.apache.org/jira/browse/FLINK-24234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Danny Cranmer resolved FLINK-24234.
-----------------------------------
Resolution: Done
> [FLIP-171] Byte Based & Time Based Flushing for AsyncSinkBase
> -------------------------------------------------------------
>
> Key: FLINK-24234
> URL: https://issues.apache.org/jira/browse/FLINK-24234
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Common
> Reporter: Zichen Liu
> Assignee: Zichen Liu
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> *User stories*
> * As a Sink user, I’d like to configure the batch size for items to send to the destination at once (e.g. “flush if there are x number of items in the batch”)
> * As a Sink user, I’d like to configure the batching logic so that I can flush the batch of requests based on time period (e.g. “flush every 2 seconds”)
> * As a Sink user I’d like to specify the number of bytes for the batch of requests to be flushed (e.g. ”submit the batch after the total number of bytes in it is above 1KB”)
> * As a Sink developer, I’d like to use the configuration mechanism provided to allow Sink users to configure my Sink implementation
> *
> *Scope*
> * Allow Sink developers and users to pass batch size config to the AsyncSinkWriter
> * Add support for time-based flushing (e.g. “flush after x miliseconds”) using the ProcessingTimeService which is part of the Sink interface
> * Add support for byte-based flushing
> * Consider the combination of time-based flushing and byte-based flushing, if there are more bytes than configured in the time-based batch, then the last few (however many necessary) items should go in the next batch to satisfy the requirement for the number of bytes.
> *References*
> More details to be found [https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)