kafka consumer coordinator is unknown issue

["Ashwini Mhatre (asmhatre)" <as...@cisco.com>]

Hi,
I have 3 node kafka cluster.
I have one  kafka consumer which will listen to particular topic.
Facing following issue:

08:52:25.733 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency

08:52:25.733 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency

08:52:25.734 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency

08:52:25.739 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency

08:52:25.746 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched

08:52:25.747 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched

08:52:25.747 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency

08:52:25.747 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag

08:52:25.747 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time

08:52:25.748 [SparkApp-akka.actor.default-dispatcher-2] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration autooffset.reset = earliest was supplied but isn't a known config.

08:52:25.748 [SparkApp-akka.actor.default-dispatcher-2] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration auto.commit = true was supplied but isn't a known config.

08:52:25.749 [SparkApp-akka.actor.default-dispatcher-2] INFO  o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.0.1

08:52:25.749 [SparkApp-akka.actor.default-dispatcher-2] INFO  o.a.kafka.common.utils.AppInfoParser - Kafka commitId : a7a17cdec9eaa6c5

08:52:25.750 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created

08:52:25.750 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.k.clients.consumer.KafkaConsumer - Subscribed to partition(s): orchestrator-0

08:52:25.751 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.k.clients.consumer.KafkaConsumer - Seeking to offset 0 for partition orchestrator-0

08:52:26.756 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending coordinator request for group NewCustomers to broker datapipeline-us-west-2-kafka-02.sse.com:9092 (id: -2 rack: null)

08:52:26.773 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -2 at datapipeline-us-west-2-kafka-02.sse.com:9092.

08:52:26.827 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.apache.kafka.clients.NetworkClient - Initialize connection to node -3 for sending metadata request

08:52:26.827 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -3 at datapipeline-us-west-2-kafka-03.sse.com:9092.

08:52:26.929 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.apache.kafka.clients.NetworkClient - Initialize connection to node -1 for sending metadata request

08:52:26.929 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at datapipeline-us-west-2-kafka-01.sse.com:9092.

08:52:30.739 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:30.838 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:30.938 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:31.039 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:31.139 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:31.239 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:31.340 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:31.439 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:31.539 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:31.640 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:31.740 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown

08:52:31.840 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown



Please let me know if anyone come across above issue


Regards,

Ashwini