You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Viktor Loosz (Jira)" <ji...@apache.org> on 2019/10/08 13:03:00 UTC
[jira] [Commented] (KAFKA-7656) ReplicaManager fetch fails on
leader due to long/integer overflow
[ https://issues.apache.org/jira/browse/KAFKA-7656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946824#comment-16946824 ]
Viktor Loosz commented on KAFKA-7656:
-------------------------------------
Dear Kafka people,
We started to hit this issue with one of our brokers today.
Kafka version: *2.0.0* (stock Kafka, not confluent)
**Protocol version: *2.0-IV1*
Log format: *2.0-IV1*
Amazon Linux 1, kernel version: 4.14.138-89.102.amzn1.x86_64
{noformat}
[2019-10-08 11:43:22,483] ERROR [ReplicaManager broker=60] Error processing fetch operation on partition __consumer_offsets-11, offset 523320867 (kafka.server.ReplicaManager)
java.lang.IllegalArgumentException: Invalid max size -2147483648 for log read from segment FileRecords(file= /var/lib/kafka/data/__consumer_offsets-11/00000000000000000000.log, start=0, end=2147483647)
at kafka.log.LogSegment.read(LogSegment.scala:274)
at kafka.log.Log.$anonfun$read$2(Log.scala:1159)
at kafka.log.Log.maybeHandleIOException(Log.scala:1837)
at kafka.log.Log.read(Log.scala:1114)
at kafka.server.ReplicaManager.read$1(ReplicaManager.scala:912)
at kafka.server.ReplicaManager.$anonfun$readFromLocalLog$6(ReplicaManager.scala:974)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:52)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at kafka.server.ReplicaManager.readFromLocalLog(ReplicaManager.scala:973)
at kafka.server.ReplicaManager.readFromLog$1(ReplicaManager.scala:810)
at kafka.server.ReplicaManager.fetchMessages(ReplicaManager.scala:815)
at kafka.server.KafkaApis.handleFetchRequest(KafkaApis.scala:687)
at kafka.server.KafkaApis.handle(KafkaApis.scala:107)
at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:69)
at java.lang.Thread.run(Thread.java:748) {noformat}
I would like to know if there's a workaround (the mentioned PR was closed without merging in August) or a patch which can be used for this Kafka version. I would update but people said in the comments that even 2.2 (and 2.1.1) is affected.
Let me know if I can help with logs or anything.
Thanks!
> ReplicaManager fetch fails on leader due to long/integer overflow
> -----------------------------------------------------------------
>
> Key: KAFKA-7656
> URL: https://issues.apache.org/jira/browse/KAFKA-7656
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 2.0.1
> Environment: Linux 3.10.0-693.el7.x86_64 #1 SMP Thu Jul 6 19:56:57 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
> Reporter: Patrick Haas
> Assignee: Jose Armando Garcia Sancio
> Priority: Major
>
> (Note: From 2.0.1-cp1 from confluent distribution)
> {{[2018-11-19 21:13:13,687] ERROR [ReplicaManager broker=103] Error processing fetch operation on partition __consumer_offsets-20, offset 0 (kafka.server.ReplicaManager)}}
> {{java.lang.IllegalArgumentException: Invalid max size -2147483648 for log read from segment FileRecords(file= /prod/kafka/data/kafka-logs/__consumer_offsets-20/00000000000000000000.log, start=0, end=2147483647)}}
> {{ at kafka.log.LogSegment.read(LogSegment.scala:274)}}
> {{ at kafka.log.Log$$anonfun$read$2.apply(Log.scala:1159)}}
> {{ at kafka.log.Log$$anonfun$read$2.apply(Log.scala:1114)}}
> {{ at kafka.log.Log.maybeHandleIOException(Log.scala:1842)}}
> {{ at kafka.log.Log.read(Log.scala:1114)}}
> {{ at kafka.server.ReplicaManager.kafka$server$ReplicaManager$$read$1(ReplicaManager.scala:912)}}
> {{ at kafka.server.ReplicaManager$$anonfun$readFromLocalLog$1.apply(ReplicaManager.scala:974)}}
> {{ at kafka.server.ReplicaManager$$anonfun$readFromLocalLog$1.apply(ReplicaManager.scala:973)}}
> {{ at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)}}
> {{ at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)}}
> {{ at kafka.server.ReplicaManager.readFromLocalLog(ReplicaManager.scala:973)}}
> {{ at kafka.server.ReplicaManager.readFromLog$1(ReplicaManager.scala:802)}}
> {{ at kafka.server.ReplicaManager.fetchMessages(ReplicaManager.scala:815)}}
> {{ at kafka.server.KafkaApis.handleFetchRequest(KafkaApis.scala:685)}}
> {{ at kafka.server.KafkaApis.handle(KafkaApis.scala:114)}}
> {{ at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:69)}}
> {{ at java.lang.Thread.run(Thread.java:748)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)