You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Tayida Tapjinda (JIRA)" <ji...@apache.org> on 2018/10/06 14:00:00 UTC
[jira] [Commented] (KAFKA-6610) initial high watermark -1, used for
truncation, cause "Cannot truncate to a negative offset"
[ https://issues.apache.org/jira/browse/KAFKA-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16640728#comment-16640728 ]
Tayida Tapjinda commented on KAFKA-6610:
----------------------------------------
Just in case if some one is on the old version and facing this issue.
This is how we recover.
* Stop the broker
* Manually update replication-offset-checkpoint file (this should be in your data directory)
** find topics with -1 offset
** replace -1 with 0
** delete all data for those topics
* Start the broker up and the ReplicaFetcherThread should be able to operate as normal
> initial high watermark -1, used for truncation, cause "Cannot truncate to a negative offset"
> --------------------------------------------------------------------------------------------
>
> Key: KAFKA-6610
> URL: https://issues.apache.org/jira/browse/KAFKA-6610
> Project: Kafka
> Issue Type: Bug
> Components: log, replication
> Affects Versions: 0.11.0.2
> Reporter: dongyan li
> Priority: Major
>
> Hello,
> I got issue that cause the Kafka broker not in-sync with topics. When I check the log, found one of the topic has "-1" highwatermark, then used as the offset for truncating and later caused exception:
> {quote}3/5/2018 10:32:26 AM[2018-03-05 16:32:26,576] ERROR [ReplicaFetcherThread-0-1]: Error due to (kafka.server.ReplicaFetcherThread)
> 3/5/2018 10:32:26 AMjava.lang.IllegalArgumentException: Cannot truncate to a negative offset (-1).
> 3/5/2018 10:32:26 AM at kafka.log.Log.truncateTo(Log.scala:1377)
> 3/5/2018 10:32:26 AM at kafka.log.LogManager.$anonfun$truncateTo$2(LogManager.scala:330)
> 3/5/2018 10:32:26 AM at kafka.log.LogManager.$anonfun$truncateTo$2$adapted(LogManager.scala:321)
> 3/5/2018 10:32:26 AM at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:789)
> 3/5/2018 10:32:26 AM at scala.collection.mutable.HashMap.$anonfun$foreach$1(HashMap.scala:138)
> 3/5/2018 10:32:26 AM at scala.collection.mutable.HashTable.foreachEntry(HashTable.scala:236)
> 3/5/2018 10:32:26 AM at scala.collection.mutable.HashTable.foreachEntry$(HashTable.scala:229)
> 3/5/2018 10:32:26 AM at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
> 3/5/2018 10:32:26 AM at scala.collection.mutable.HashMap.foreach(HashMap.scala:138)
> 3/5/2018 10:32:26 AM at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:788)
> 3/5/2018 10:32:26 AM at kafka.log.LogManager.truncateTo(LogManager.scala:321)
> 3/5/2018 10:32:26 AM at kafka.server.ReplicaFetcherThread.maybeTruncate(ReplicaFetcherThread.scala:279)
> 3/5/2018 10:32:26 AM at kafka.server.AbstractFetcherThread.$anonfun$maybeTruncate$2(AbstractFetcherThread.scala:133)
> 3/5/2018 10:32:26 AM at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
> 3/5/2018 10:32:26 AM at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:213)
> 3/5/2018 10:32:26 AM at kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
> 3/5/2018 10:32:26 AM at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:102)
> 3/5/2018 10:32:26 AM at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64)
> {quote}
> I deleted the log directory of that topic but topic partition got re-created after restart, with same error after. I'm wondering how the `highwatermark` get set to -1. I failed to trace the code flow from the `Replica` constructor. Thank you.
>
> Dongyan
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)