You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@nifi.apache.org by "Pierre Villard (JIRA)" <ji...@apache.org> on 2018/03/12 20:03:00 UTC

[jira] [Created] (NIFI-4964) Add bulk lookup feature in LookupRecord

Pierre Villard created NIFI-4964:
------------------------------------

             Summary: Add bulk lookup feature in LookupRecord
                 Key: NIFI-4964
                 URL: https://issues.apache.org/jira/browse/NIFI-4964
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Extensions
            Reporter: Pierre Villard


When having a flow file with a large number of records it would be much more efficient to parse the whole flow file once to list all the coordinates to look for, then call a new method (lookupAll?) in the lookup service to get all the results, and then parse the file one more time to update the records.

It should be added in the CS description/annotations that this approach could hold in memory a large number of objects but could result in better performances for lookup services accessing external systems (Mongo, HBase, etc).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)