[jira] [Updated] (NIFI-10737) ListenBeats Unable to Process Large Batches

["David Handermann (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-10737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Handermann updated NIFI-10737:
------------------------------------
    Status: Patch Available  (was: In Progress)

> ListenBeats Unable to Process Large Batches
> -------------------------------------------
>
>                 Key: NIFI-10737
>                 URL: https://issues.apache.org/jira/browse/NIFI-10737
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.18.0, 1.17.0, 1.16.0
>            Reporter: David Handermann
>            Assignee: David Handermann
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Refactoring of ListenBeats released in version 1.16.0 introduced processing issues when reading batches of messages larger than 2048 bytes.
> The refactored implementation based on Netty introduced a {{BeatsFrameDecoder}} extension of the Netty {{{}ByteToMessageDecoder{}}}, which the Netty framework invokes multiple times with variables contents of the input {{{}ByteBuf{}}}. The implementation does not handle scenarios where a batch of multiple messages is larger than the buffer size of 2048. As a result of this behavior, the Decoder does not construct a complete message. Without the complete message, the Processor does not send an acknowledgement for the received message, resulting in errors on sending clients, such as the following:
> {noformat}
> Failed to publish events caused by: read tcp 192.168.1.100:54623->192.168.1.200:5555: i/o timeout
> {noformat}
> This error occurs on sending client while waiting to receive an acknowledgement packet for the messages transmitted.
> As a result of not receiving the acknowledgement, the client attempts to send the same messages multiple times, resulting in duplication and failure to send additional messages.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)