You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@nifi.apache.org by "Gábor Gyimesi (Jira)" <ji...@apache.org> on 2022/10/20 08:51:00 UTC

[jira] [Created] (MINIFICPP-1967) Add batch processing of lines in TailFile

Gábor Gyimesi created MINIFICPP-1967:
----------------------------------------

             Summary: Add batch processing of lines in TailFile
                 Key: MINIFICPP-1967
                 URL: https://issues.apache.org/jira/browse/MINIFICPP-1967
             Project: Apache NiFi MiNiFi C++
          Issue Type: Improvement
            Reporter: Gábor Gyimesi


When TailFile starts to process a new file it processes the whole file with all its lines in a single onTrigger call. Due to this if a large file is presented it results in lot of memory usage for a short time to process all the flow files at once. We should add an option to maximize the number of flow files generated in a single onTrigger call to lower the memory consumption at a single point of time.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)