[jira] [Created] (NIFI-8689) Site-to-Site client is constantly flushing the socket's OutputStream

["Mark Payne (Jira)" <ji...@apache.org>]

Mark Payne created NIFI-8689:
--------------------------------

             Summary: Site-to-Site client is constantly flushing the socket's OutputStream
                 Key: NIFI-8689
                 URL: https://issues.apache.org/jira/browse/NIFI-8689
             Project: Apache NiFi
          Issue Type: Bug
          Components: Core Framework
            Reporter: Mark Payne
            Assignee: Mark Payne


When a RemoteProcessGroup is sending data to another NiFi instance, the protocol should establish a transaction and then send a sequence of FlowFiles following a pattern along the lines of:
{code:java}
<FlowFile Follows><FlowFile Attributes><FlowFile Content>
<FlowFile Follows><FlowFile Attributes><FlowFile Content>
<FlowFile Follows><FlowFile Attributes><FlowFile Content>
<Finished Transaction>{code}
However, currently, the protocol is flushing the Socket's output buffer each that that it indicates that a FlowFile follows, and again after each FlowFile. So it's more like:
{code:java}
<FlowFile Follows>*Flush Buffer*
<FlowFile Attributes><FlowFile Content>*Flush Buffer*
<FlowFile Follows>*Flush Buffer*
<FlowFile Attributes><FlowFile Content>*Flush Buffer*
<FlowFile Follows>*Flush Buffer*
<FlowFile Attributes><FlowFile Content>*Flush Buffer*{code}
As a result, when sending a large number of smaller FlowFiles, we end up constantly flushing data to the socket, which results in dramatically worse performance.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)