You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@nifi.apache.org by "Mark Heydenrych (JIRA)" <ji...@apache.org> on 2017/07/25 10:57:00 UTC

[jira] [Created] (NIFI-4223) Batching in ListFile processor

Mark Heydenrych created NIFI-4223:
-------------------------------------

             Summary: Batching in ListFile processor
                 Key: NIFI-4223
                 URL: https://issues.apache.org/jira/browse/NIFI-4223
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Configuration
    Affects Versions: 1.0.0
            Reporter: Mark Heydenrych
            Priority: Minor


The ListFile processor builds a full list of all files before sending any flowfiles into the queue to be processed. When the number of files is exceptionally large, building the list takes a very long time and appears to be hanging. It would be best if we could set a batch size e.g. 1000 files, and have the ListFile processor stop listing after the reaching this many files, send the files through to the queue and then resume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)