You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Paul Grey (Jira)" <ji...@apache.org> on 2024/08/13 14:39:00 UTC

[jira] [Created] (NIFI-13654) ConsumeKafka: implement static partition assignment

Paul Grey created NIFI-13654:
--------------------------------

             Summary: ConsumeKafka: implement static partition assignment
                 Key: NIFI-13654
                 URL: https://issues.apache.org/jira/browse/NIFI-13654
             Project: Apache NiFi
          Issue Type: Improvement
            Reporter: Paul Grey


The ConsumeKafka_2_6 processor allows for static partition assignment, as described in the additional details section of the processor usage documentation:

[https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-kafka-2-6-nar/1.27.0/org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_2_6/additionalDetails.html]

 
{quote}In order to provide a static mapping of node to Kafka partition(s), one or more user-defined properties must be added using the naming scheme {{partitions.<hostname>}} with the value being a comma-separated list of Kafka partitions to use. For example, {{{}partitions.nifi-01=0, 3, 6, 9{}}}, {{{}partitions.nifi-02=1, 4, 7, 10{}}}, and {{{}partitions.nifi-03=2, 5, 8, 11{}}}. The hostname that is used can be the fully qualified hostname, the "simple" hostname, or the IP address. There must be an entry for each node in the cluster, or the Processor will become invalid. If it is desirable for a node to not have any partitions assigned to it, a Property may be added for the hostname with an empty string as the value.{quote}

Evaluate feature for potential inclusion in the refactored nifi2/ConsumeKafka processor.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)