[jira] [Commented] (NIFI-6677) ListHDFS can clear state and re-list everything on Primary Node change if property was modified before Processor started

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931486#comment-16931486 ] 

ASF subversion and git services commented on NIFI-6677:
-------------------------------------------------------

Commit fd3f0707c66b78622ed4e7f5aeaff63580c46d44 in nifi's branch refs/heads/master from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=fd3f070 ]

NIFI-6677: Update ListHDFS to clear state (when appropriate) in an @OnScheduled method, just as AbstractListProcessor does, instead of doing it in onTrigger. Doing it in onTrigger is problematic because in a cluster, the Primary Node may run for some period of time, perhaps days or months. Then, when the Primary Node chagnes, onTrigger gets called for the first time on the new Primary Node, and this triggers the processor to clear state.


> ListHDFS can clear state and re-list everything on Primary Node change if property was modified before Processor started
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-6677
>                 URL: https://issues.apache.org/jira/browse/NIFI-6677
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Major
>             Fix For: 1.10.0
>
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> If the "Directory" or "File Filter" property is changed in ListHDFS, and then the Processor is started, a change in the Primary Node can result in resetting the Processor state.
> To replicate, create a ListHDFS Processor in a cluster. Configure to point to some directory and start the processor running on Primary Node. After the listing has completed, disconnect the Primary Node from the cluster to trigger the Primary Node to change. At this point, the Processor will reset its state and re-list everything.
> Note that once NiFi has been restarted, this behavior will no longer exist until the Processor's Directory or "File Filter" property is changed again. It also will not happen again if the new Primary Node has already reset the state as a result of the property change.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)