[jira] [Assigned] (NIFI-2835) GetAzureEventHub processor should leverage partition offset to better handle restarts

["Joseph Niemiec (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joseph Niemiec reassigned NIFI-2835:
------------------------------------

    Assignee: Joseph Niemiec

> GetAzureEventHub processor should leverage partition offset to better handle restarts
> -------------------------------------------------------------------------------------
>
>                 Key: NIFI-2835
>                 URL: https://issues.apache.org/jira/browse/NIFI-2835
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Joseph Percivall
>            Assignee: Joseph Niemiec
>
> The GetAzureEventHub processor utilizes the Azure client that consists of receivers for each partition. The processor stores them in a map[1] that gets cleared every time the processor is stopped[2]. These receivers have partition offsets which keep track of which message it's currently on and which it should receive next. So currently, when the processor is stopped/restarted, any tracking of which message is next to be received is lost.
> If instead of clearing the map each time, we hold onto the receivers, or kept track of the partitionId/Offsets when stopping, (barring any relevant configuration changes) the processor would restart exactly where it left off with no loss of data.
> This would work very well with NIFI-2826.
> [1]https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/GetAzureEventHub.java#L122
> [2] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/GetAzureEventHub.java#L229



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)