You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Guozhang Wang (JIRA)" <ji...@apache.org> on 2014/10/07 00:22:34 UTC

[jira] [Commented] (KAFKA-1641) Log cleaner exits if last cleaned offset is lower than earliest offset

    [ https://issues.apache.org/jira/browse/KAFKA-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161121#comment-14161121 ] 

Guozhang Wang commented on KAFKA-1641:
--------------------------------------

Created reviewboard https://reviews.apache.org/r/26390/diff/
 against branch origin/trunk

> Log cleaner exits if last cleaned offset is lower than earliest offset
> ----------------------------------------------------------------------
>
>                 Key: KAFKA-1641
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1641
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8.1.1
>            Reporter: Joel Koshy
>         Attachments: KAFKA-1641.patch
>
>
> Encountered this recently: the log cleaner exited a while ago (I think because the topic had compressed messages). That issue was subsequently addressed by having the producer only send uncompressed. However, on a subsequent restart of the broker we see this:
> In this scenario I think it is reasonable to just emit a warning and have the cleaner round up its first dirty offset to the base offset of the first segment.
> {code}
> [kafka-server] [] [kafka-log-cleaner-thread-0], Error due to 
> java.lang.IllegalArgumentException: requirement failed: Last clean offset is 54770438 but segment base offset is 382844024 for log testtopic-0.
>         at scala.Predef$.require(Predef.scala:145)
>         at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:491)
>         at kafka.log.Cleaner.clean(LogCleaner.scala:288)
>         at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:202)
>         at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:187)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)