[jira] [Resolved] (NIFI-8021) Allow ConsumeKafka processors to use static partition mapping

["Pierre Villard (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-8021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pierre Villard resolved NIFI-8021.
----------------------------------
    Fix Version/s: 1.13.0
       Resolution: Fixed

> Allow ConsumeKafka processors to use static partition mapping
> -------------------------------------------------------------
>
>                 Key: NIFI-8021
>                 URL: https://issues.apache.org/jira/browse/NIFI-8021
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Major
>             Fix For: 1.13.0
>
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> With the ConsumeKafka processors, we use the KafkaConsumer.subscribe() method, which is generally what is desirable. However, for a case where strict FIFO order per partition is critical, as may be the case with CDC use cases among others, we should allow users to pin a particular partition to a particular node. This way, even if the node is restarted or stops pulling data for a while, its partitions are not re-assigned, so there's no risk of messages being delivered out of order.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)