You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Manikumar (JIRA)" <ji...@apache.org> on 2018/05/18 09:02:00 UTC
[jira] [Resolved] (KAFKA-3843) Endless consuming messages in Kafka
cluster
[ https://issues.apache.org/jira/browse/KAFKA-3843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manikumar resolved KAFKA-3843.
------------------------------
Resolution: Auto Closed
The Scala consumers have been deprecated and no further work is planned, please upgrade to the Java consumer whenever possible.
> Endless consuming messages in Kafka cluster
> -------------------------------------------
>
> Key: KAFKA-3843
> URL: https://issues.apache.org/jira/browse/KAFKA-3843
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.9.0.1
> Reporter: Tomas Benc
> Assignee: Neha Narkhede
> Priority: Major
>
> We are running Kafka in cluster (3 virtual machines). Kafka is configured min.insync.replicas = 2 and topics are configured replication factor = 3. This configuration means, there must be at least 2 brokers of 3 in cluster up and running to receive any messages. This works as expected.
> Our consumers are high level consumers and offsets are commited manually (auto.commit disabled) and stored in Kafka.
> Reproducing the issue:
> 1. Kafka cluster up and running and receives messages
> 2. Consumers are disabled (messages in Kafka are in lag)
> 3. Disable 2 Kafka brokers in cluster
> 4. Enable consumers
> Consumers are consuming messages in batch, and commiting offsets after processing. But commit offsets fails in Kafka, because of NotEnoughReplicasException. That is correct. What is not correct, high level consumer has no idea, that offset are not commited and consumes same messages again and again.
> It would be helpful, that method commitOffsets() in interface kafka.javaapi.consumer.ConsumerConnector should return some information (return boolean or throw exception) about this operation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)