[jira] [Comment Edited] (NIFI-12809) PublishKafkaRecord_2_6 - RoundRobin partitioner skipping every other partition

["Joe Witt (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17818086#comment-17818086 ] 

Joe Witt edited comment on NIFI-12809 at 2/16/24 7:56 PM:
----------------------------------------------------------

[~slyouts] Did you review any Kafka bugs for the noted scenario?  I see https://issues.apache.org/jira/browse/KAFKA-13180 that sounds very closely related.

If there is a reasonable workaround for this issue that we can control on the NiFi side we should ensure we remove any such change in versions it is resolved in (such as Kafka 3 components/etc..) 



was (Author: joewitt):
[~slyouts] Did you review any Kafka bugs for the noted scenario?  I see https://issues.apache.org/jira/browse/KAFKA-13180 that sounds very closely related.


> PublishKafkaRecord_2_6 - RoundRobin partitioner skipping every other partition
> ------------------------------------------------------------------------------
>
>                 Key: NIFI-12809
>                 URL: https://issues.apache.org/jira/browse/NIFI-12809
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.25.0
>            Reporter: Steven Youtsey
>            Priority: Major
>              Labels: kafka, partitioners, publish
>
> When configured to use the NiFi RoundRobin partitioner, the processor publishes to every other partition. If the number of partitions in the topic and the number of records being published are the right combination, this problem is masked. We see this issue when we set the partitions to 26, but not when set to 25. 
> I took a code-dive into the o.a.k.c.producer.KafkaProducer and discovered that it is invoking the Partitioner twice when a "new batch" is created. Thus, the RoundRobin partitioner bumps the index by 2. If the RoundRobin partitioner overwrote the onNewBatch method, this problem could be solved.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)