[jira] [Assigned] (KAFKA-12261) Splitting partition causes message loss for consumers with auto.offset.reset=latest

["Luke Chen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Luke Chen reassigned KAFKA-12261:
---------------------------------

    Assignee: Luke Chen

> Splitting partition causes message loss for consumers with auto.offset.reset=latest
> -----------------------------------------------------------------------------------
>
>                 Key: KAFKA-12261
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12261
>             Project: Kafka
>          Issue Type: Improvement
>    Affects Versions: 2.7.0
>            Reporter: Haruki Okada
>            Assignee: Luke Chen
>            Priority: Minor
>
> As of now, auto.offset.reset of ConsumerConfig is "latest" by default.
>  
> This could be a pitfall that causes message delivery loss when we split topic's partitions like below:
> Say we have a topic-X which have only 1 partition.
>  # split topic-X to 2 partitions by kafka-topics.sh --alter --topic topic-X --partitions 2 (topic-X-1 is added)
>  # producer knows that new partitions are added by refreshing metadata. starts to produce to topic-X-1
>  # bit later, consumer knows that new partitions are added and triggering consumer rebalance, then starts consuming topic-X-1
>  * 
>  ** upon starting consumption, it resets its offset to log-end-offset
> If the producer sent several records before 3, they could be not-delivered to the consumer.
>  
>  
> This behavior isn't preferable in most cases, so auto.offset.reset should be set to "earliest" by default to avoid this pitfall.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)