You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Matthew Rathbone <ma...@foursquare.com> on 2013/04/13 19:03:14 UTC

We have an invalid message on a broker, how to deal with it?

Hey guys,

Due to a disk filling up, one of the segments has an invalid message in it.
I have verified this using DumpLogSegments.

How do I deal with this now? the invalid message is causing our Hadoop
Consumer to fail.

Is there a way to remove the invalid message from the segment? Removing the
bad segment causes the broker to fail on startup with an error:

java.lang.IllegalStateException: The following segments don't validate:
<bad-file>, <bad-file+1>

I'm happy losing that file if needs be, but I need to get this broker back
up asap.


-- 
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
matthew@foursquare.com | @rathboma <http://twitter.com/rathboma> |
4sq<http://foursquare.com/rathboma>