[jira] [Commented] (NIFI-1575) Add ability to query DB table using "last value" from column

["Randy Gelhausen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-1575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172089#comment-15172089 ] 

Randy Gelhausen commented on NIFI-1575:
---------------------------------------

I see use cases where the query needs to run not based exclusively on monotonically increasing values, but on some context provided by an external trigger.

Thus a very useful (and hopefully not too complex) additional feature would be to allow the user to pull the "last value" from DistributedMapCache. This way external flows/activities may act as a more customizable and fine-grained control mechanism for the QueryDBTable processor.

> Add ability to query DB table using "last value" from column
> ------------------------------------------------------------
>
>                 Key: NIFI-1575
>                 URL: https://issues.apache.org/jira/browse/NIFI-1575
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Matt Burgess
>            Assignee: Matt Burgess
>
> It would be a useful feature for a processor to be able to query from a database table for only those records that have are added/available since the last time the query was executed. For example, if the processor could keep the max value of a timestamp column for the last result set returned, the next time the query was issued, it could use that timestamp value and column to only return records whose timestamps were greater than "the last time".
> This shouldn't be limited to timestamps of course; it would be useful for any strictly-increasing value (like primary key ID) but would be up to the user to select the table and column. The processor would simply keep the state for the specified column's max value.
> Proposed is a "QueryDBTable" processor which would have a properties to specify the table name and the column in the table for which to keep state information about the last maximum value retrieved. Subsequent queries of the table use this value to filter for rows whose value for the specified column are greater than the "last maximum value". Upon each successful query, the "last maximum value" is updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)