[jira] [Commented] (KAFKA-7358) Alternative Partitioner to Support "Always Round-Robin" Selection

["M. Manna (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-7358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843951#comment-16843951 ] 

M. Manna commented on KAFKA-7358:
---------------------------------

[~mjsax] Yes - looks like my pull request effort was not put in the correct place. Could we please consider 7358 for this since a KIP and dev work have been put into this?

> Alternative Partitioner to Support "Always Round-Robin" Selection
> -----------------------------------------------------------------
>
>                 Key: KAFKA-7358
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7358
>             Project: Kafka
>          Issue Type: Wish
>          Components: clients
>            Reporter: M. Manna
>            Assignee: M. Manna
>            Priority: Minor
>
> In my organisation, we have been using kafka as the basic publish-subscribe messaging system provider. Our goal is the send event-based (secure, encrypted) SQL messages reliably, and process them accordingly. For us, the message keys represent some metadata which we use to either ignore messages (if a loopback to the sender), or log some information. We have the following use case for messaging:
> 1) A Database transaction event takes place
> 2) The event is captured and messaged across 10 data centres all around the world.
> 3) A group of consumers (for each data centre with a unique consumer-group ID) are will process messages from their respective partitions. 1 consumer per partition.
> Under the circumstances, we only need a guarantee that same message won't be sent to multiple partitions. In other words, 1 partition will +never+ be sought by multiple consumers.
> Using DefaultPartitioner, we can achieve this only with NULL keys. But since we need keys for metadata, we cannot maintain "Round-robin" selection of partitions because a key hash will determine which partition to choose. We need to have round-robin style selection regardless of key type (NULL or not-NULL)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)