You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Manikumar (JIRA)" <ji...@apache.org> on 2018/05/18 15:36:00 UTC
[jira] [Resolved] (KAFKA-5728) Stopping consumer thread cause
loosing message in the partition
[ https://issues.apache.org/jira/browse/KAFKA-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manikumar resolved KAFKA-5728.
------------------------------
Resolution: Auto Closed
looks like this is related to spring kafka config issue. must be related to committing offsets. Pls take a look at spring kafka docs.
> Stopping consumer thread cause loosing message in the partition
> ---------------------------------------------------------------
>
> Key: KAFKA-5728
> URL: https://issues.apache.org/jira/browse/KAFKA-5728
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 0.10.1.0
> Reporter: Vasudevan Karnan
> Priority: Major
>
> Currently using Spring boot Kafka listener thread to consume the message from partition.
> Having 10 partitions and concurrency to 10 in the consumer group.
> In testing, I have 2 messages in the single partition (say for ex: partition 4). Created listener to read the message and post to service. During normal days, read the message and post to service, and working as expected. No issues on that.
> Suppose if the service is down, then I am doing Spring Retry template to retry to post the message to service (repeatedly) for number of retry and backoff time in ms. If I stop the listener, then getting
> org.springframework.retry.backoff.BackOffInterruptedException: Thread interrupted while sleeping; nested exception is java.lang.InterruptedException: sleep interrupted
> at org.springframework.retry.backoff.FixedBackOffPolicy.doBackOff(FixedBackOffPolicy.java:86) ~[spring-retry-1.1.4.RELEASE.jar:na]
> at org.springframework.retry.backoff.StatelessBackOffPolicy.backOff(StatelessBackOffPolicy.java:36) ~[spring-retry-1.1.4.RELEASE.jar:na]
> After that I am loosing the message from particular partition (message that are got retried is lost in the middle) and lag is reduced. (This is happening during the end of stopping the listener).
> Is there any way, we don't loose the message even I am getting the sleep interrupted exception?
> Suppose during stopping the server, if I dont face sleep interrupt exception, in the next time listener startup, face the same issue and loosing the message again.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)