[jira] [Resolved] (KAFKA-4091) Unable to produce or consume on any topic

["Manikumar (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Manikumar resolved KAFKA-4091.
------------------------------
    Resolution: Cannot Reproduce

{color:#000000}Closing inactive issue. Please reopen if you think the issue still exists.{color}
 

> Unable to produce or consume on any topic
> -----------------------------------------
>
>                 Key: KAFKA-4091
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4091
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.10.0.0
>         Environment: Amazon Linux, t2.micro
>            Reporter: Avi Chopra
>            Priority: Critical
>
> While trying to set kafka on 2 slave and 1 master box, got a weird condition where I was not able to consume or produce to a topic.
> Using Mirror Maker to sync data between slave <--> Master. Getting following logs unending :
> [2016-08-26 14:28:33,897] WARN Bootstrap broker localhost:9092 disconnected (org.apache.kafka.clients.NetworkClient) [2016-08-26 14:28:43,515] WARN Bootstrap broker localhost:9092 disconnected (org.apache.kafka.clients.NetworkClient) [2016-08-26 14:28:45,118] WARN Bootstrap broker localhost:9092 disconnected (org.apache.kafka.clients.NetworkClient) [2016-08-26 14:28:46,721] WARN Bootstrap broker localhost:9092 disconnected (org.apache.kafka.clients.NetworkClient) [2016-08-26 14:28:48,324] WARN Bootstrap broker localhost:9092 disconnected (org.apache.kafka.clients.NetworkClient) [2016-08-26 14:28:49,927] WARN Bootstrap broker localhost:9092 disconnected (org.apache.kafka.clients.NetworkClient) [2016-08-26 14:28:53,029] WARN Bootstrap broker localhost:9092 disconnected (org.apache.kafka.clients.NetworkClient)
> Only way I could recover was by restarting Kafka which produced this kind of logs :
> [2016-08-26 14:30:54,856] WARN Found a corrupted index file, /tmp/kafka-logs/__consumer_offsets-43/00000000000000000000.index, deleting and rebuilding index... (kafka.log.Log) [2016-08-26 14:30:54,856] INFO Recovering unflushed segment 0 in log __consumer_offsets-43. (kafka.log.Log) [2016-08-26 14:30:54,857] INFO Completed load of log __consumer_offsets-43 with log end offset 0 (kafka.log.Log) [2016-08-26 14:30:54,860] WARN Found a corrupted index file, /tmp/kafka-logs/__consumer_offsets-26/00000000000000000000.index, deleting and rebuilding index... (kafka.log.Log) [2016-08-26 14:30:54,860] INFO Recovering unflushed segment 0 in log __consumer_offsets-26. (kafka.log.Log) [2016-08-26 14:30:54,861] INFO Completed load of log __consumer_offsets-26 with log end offset 0 (kafka.log.Log) [2016-08-26 14:30:54,864] WARN Found a corrupted index file, /tmp/kafka-logs/__consumer_offsets-35/00000000000000000000.index, deleting and rebuilding index... (kafka.log.Log)
> ERROR Error when sending message to topic dr_ubr_analytics_limits with key: null, value: 1 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
> The consumer group command was showing a major lag.
> This is my test phase so I was able to restart and recover from the master box but I want know what caused this issue and how can it be avoided. Is there a way to debug this issue?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)