Regarding batch settings in RPG

["Tirumanyam, Dharani" <dt...@blueplanet.com.INVALID>]

Hi Nifi Team,

I am reaching out to you for the information regarding batch settings of Remote process group.
Please find the below details about Nifi and system setup we are using:
Nifi version: 1.21.0
set up: 3 node cluster for remote and client with 1TB storage.

We established communication between client and server/remote Nifi using remote process group using HTTP transport protocol which is working fine. We have 2 input ports and one output port in client Nifi from where we are sending/receiving flow files to/from the remote Nifi. Here when we are trying to send/recieve the flow files to/from remote Nifi we observed a delay which is approximately 5 seconds sometimes it will be lesser or greater than 5 seconds (While receiving flow files from input ports delay is less than 5 seconds). In our use case Nifi should not have that much delay we are expecting flow files will send/receive within few milli seconds. We have tested using default batch settings and observed delay as default batch settings of output port duration is 5 seconds and input port duration is 500 Millis, so went through Nifi document to know how to configure the batch settings to avoid delay.

We have verified the Nifi documents and Apache Nifi jira and blogs regarding RPG configurations there is no proper information about how batch settings (i.e., count, size, and duration) should be configured. Tested by configuring only duration and it didn’t helped and tried by configuring count, size and duration and we have observed the behavior didn’t changed and the delay is same as like default duration (refreshed the RPG and even restarted the nifi ). This use case were tested in standalone servers also and observed the same. Please find the attachment for RPG batch settings.

Could you please help us by providing more details on this and please correct us if we are missing something and let us know if you need more details on Nifi configurations in our environment.


Thanks and Regards,
Dharani T.