[jira] [Created] (NIFI-5519) Allow ListDatabaseTables to accept incoming connections

["Matt Burgess (JIRA)" <ji...@apache.org>]

Matt Burgess created NIFI-5519:
----------------------------------

             Summary: Allow ListDatabaseTables to accept incoming connections
                 Key: NIFI-5519
                 URL: https://issues.apache.org/jira/browse/NIFI-5519
             Project: Apache NiFi
          Issue Type: Wish
            Reporter: Matt Burgess


As of [NIFI-5229|https://issues.apache.org/jira/browse/NIFI-5229], DBCPConnectionPoolLookup allows the dynamic selection of a DBCPConnectionPool by name. This allows processors who are to perform the same work on multiple databases to be able to do so by providing individual flow files upstream with the database.name attribute set.

However ListDatabaseTables does not accept incoming connections, so you currently need 1 DBCPConnectionPool per database, plus 1 ListDatabaseTables per database, each using a corresponding DBCPConnectionPool. It would be nice if ListDatabaseTables could accept incoming connection(s), if only to provide attributes for selecting the DBCPConnectionPool.

I propose the behavior be like other processors that can generate data with or without an incoming connection (such as GenerateTableFetch, see [NIFI-2881|https://issues.apache.org/jira/browse/NIFI-2881] for details). In general that means if there is an incoming non-loop connection, it becomes more "event-driven" in the sense that it will not execute if there is no FlowFile on which to work. If there is no incoming connection, then it would run as it always has, on its Run Schedule and with State Management, so as not to re-list the same tables every time it executes. 

However with an incoming connection and an available FlowFile, the behavior could be that all tables for that database are listed, meaning that processor state would not be updated nor queried, making it fully "event-driven". If the tables for a database are not to be re-listed, the onus would be on the upstream flow to not send a flow file for that database. This is not a requirement, just a suggestion; it could be more flexible by honoring processor state if the Refresh Interval is non-zero, but I think that adds too much complexity for the user, for little payoff.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)