My brokers stopped accepting events...

[Guy Doulberg <gu...@conduit.com>]

Hi All,

Yesterday, my prod kafka service has stopped accepting events for one of 
my topics,

I have 3 brokers in my cluster, and 2 producers...

My producers are reporting to 2 topics, one of which was not excepting 
events.

I saw this exception in all my  brokers, around the time of the problem, 
but I am not sure it is related:

ERROR error when processing request FetchRequest(topic:Raw, part:2 
offset:26318976078 maxSize:307200) (kafka.server.KafkaRequestHandlers)
2012-07-25_19:32:46.89759 kafka.common.OffsetOutOfRangeException: offset 
26318976078 is out of range


In order to return my service to the state of working I tried to restart 
the brokers- but it didn't help me....

Then I tried to restart the producer process, this actually helped.

Unfortunately, I can't find the logs of the producer to this hour....

Has one of you faced such a case?

Thanks Guy,

P.S
I knew there is a problem using the TopicOffsetChecker (and nagios) I 
shared with you a week ago

Re: My brokers stopped accepting events...

[Jun Rao <ju...@gmail.com>]

Guy,

The ERROR is for the fetch request, not the produce request. Any other
exception/error you saw? Any exception/error on the producer side?

Thanks,

Jun

On Thu, Jul 26, 2012 at 1:02 AM, Guy Doulberg <gu...@conduit.com>wrote:

> Hi All,
>
> Yesterday, my prod kafka service has stopped accepting events for one of
> my topics,
>
> I have 3 brokers in my cluster, and 2 producers...
>
> My producers are reporting to 2 topics, one of which was not excepting
> events.
>
> I saw this exception in all my  brokers, around the time of the problem,
> but I am not sure it is related:
>
> ERROR error when processing request FetchRequest(topic:Raw, part:2
> offset:26318976078 maxSize:307200) (kafka.server.**KafkaRequestHandlers)
> 2012-07-25_19:32:46.89759 kafka.common.**OffsetOutOfRangeException:
> offset 26318976078 is out of range
>
>
> In order to return my service to the state of working I tried to restart
> the brokers- but it didn't help me....
>
> Then I tried to restart the producer process, this actually helped.
>
> Unfortunately, I can't find the logs of the producer to this hour....
>
> Has one of you faced such a case?
>
> Thanks Guy,
>
> P.S
> I knew there is a problem using the TopicOffsetChecker (and nagios) I
> shared with you a week ago
>
>
>
>
>
>
>