You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Manikumar (JIRA)" <ji...@apache.org> on 2018/05/29 12:20:00 UTC
[jira] [Resolved] (KAFKA-2556) Corrupted index on log.dir updates
[ https://issues.apache.org/jira/browse/KAFKA-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manikumar resolved KAFKA-2556.
------------------------------
Resolution: Not A Problem
Closing inactive issue. Kafka log.dir contains topic data and metadata information. We cannot arbitrarily replace the log.dir contents. Please reopen if you think the issue still exists
> Corrupted index on log.dir updates
> ----------------------------------
>
> Key: KAFKA-2556
> URL: https://issues.apache.org/jira/browse/KAFKA-2556
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.8.2.1
> Reporter: Denis
> Priority: Major
>
> Partition is corrupted when user updates server configuration.
> Topic would be corrupted if two or more `log.dir` directories contains segments for the same topic after configuration changes.
> Steps to reproduce:
> * Start Kafka service with several `log.dir` directories
> * Stop Kafka service
> * Update configuration. Remove one of `log.dir` directories
> * Start Kafka service. Kafka creates another directory for the partition in another directory
> * Stop Kafka service
> * Update configuration. Restore the folder we removed on the third step
> * Start Kafka process
> Result:
> {code}
> [2015-09-18 10:28:48,764] INFO Verifying properties (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,801] INFO Property auto.create.topics.enable is overridden to false (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,801] INFO Property auto.leader.rebalance.enable is overridden to false (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,801] INFO Property broker.id is overridden to 6 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,802] INFO Property default.replication.factor is overridden to 1 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,802] INFO Property host.name is overridden to inferno07.chi.net (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,803] WARN Property kafka.metrics.graphite.group is not valid (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,803] WARN Property kafka.metrics.graphite.host is not valid (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,803] WARN Property kafka.metrics.graphite.port is not valid (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,803] WARN Property kafka.metrics.graphite.regex is not valid (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,803] WARN Property kafka.metrics.polling.interval.secs is not valid (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,804] WARN Property kafka.metrics.reporters is not valid (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,804] WARN Property log.cleanup.interval.mins is not valid (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,804] INFO Property log.dirs is overridden to /mnt/21e57274-0ba1-41e0-9ea4-befef32ec62c/data/kafka,/mnt/f81fc52a-7784-47c3-8b6d-be95cc935697/data/kafka,/mnt/75928c58-289d-4954-8640-0b82a990b3bc/data/kafka,/mnt/6bcb0c8e-da33-43e6-b878-703600f622d1/data/kafka,/mnt/afd7361c-dc16-4b78-8608-adda119559d0/data/kafka,/mnt/96a93811-6d91-41aa-ab1a-3804f43a5b05/data/kafka,/mnt/c085ee04-65ff-493d-b236-fa3dd462595c/data/kafka,/mnt/fbec6169-ca70-4f07-a7e9-af85424a97c5/data/kafka,/mnt/363d4473-531f-446c-b483-49593587f965/data/kafka,/mnt/d47fd346-1b77-41a8-a0a8-1c517c394868/data/kafka,/mnt/2c97da3c-4d5d-49bf-ae5a-a46c2839b18a/data/kafka,/mnt/fc6fd370-a213-4b02-9a7c-07f7a5ad505f/data/kafka (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,804] INFO Property log.retention.hours is overridden to 168 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,804] INFO Property log.segment.bytes is overridden to 536870912 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,805] INFO Property message.max.bytes is overridden to 41943040 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,805] INFO Property num.io.threads is overridden to 45 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,805] INFO Property num.network.threads is overridden to 150 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,805] INFO Property num.partitions is overridden to 1 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,806] INFO Property num.replica.fetchers is overridden to 10 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,806] INFO Property port is overridden to 9093 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,806] INFO Property replica.fetch.max.bytes is overridden to 41943040 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,806] INFO Property socket.receive.buffer.bytes is overridden to 1048576 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,806] INFO Property socket.request.max.bytes is overridden to 104857600 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,807] INFO Property socket.send.buffer.bytes is overridden to 1048576 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,808] INFO Property unclean.leader.election.enable is overridden to true (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,808] INFO Property zookeeper.connect is overridden to inferno08.chi.net:32181,inferno09.chi.net:32181,inferno10.chi.net:32181,inferno11.chi.net:32181,inferno12.chi.net:32181, (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,808] INFO Property zookeeper.connection.timeout.ms is overridden to 10000 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,808] INFO Property zookeeper.session.timeout.ms is overridden to 10000 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,809] INFO Property zookeeper.sync.time.ms is overridden to 4000 (kafka.utils.VerifiableProperties)
> [2015-09-18 10:28:48,874] INFO [Kafka Server 6], starting (kafka.server.KafkaServer)
> [2015-09-18 10:28:48,877] INFO [Kafka Server 6], Connecting to zookeeper on inferno08.chi.net:32181,inferno09.chi.net:32181,inferno10.chi.net:32181,inferno11.chi.net:32181,inferno12.chi.net:32181, (kafka.server.KafkaServer)
> [2015-09-18 10:28:49,295] INFO Loading logs. (kafka.log.LogManager)
> [2015-09-18 10:28:49,375] INFO Recovering unflushed segment 1752968 in log pony-topic-73. (kafka.log.Log)
> [2015-09-18 10:28:49,375] INFO Recovering unflushed segment 0 in log pony-topic-1. (kafka.log.Log)
> [2015-09-18 10:28:49,375] INFO Recovering unflushed segment 1753474 in log pony-topic-13. (kafka.log.Log)
> [2015-09-18 10:28:49,376] INFO Recovering unflushed segment 1753277 in log pony-topic-61. (kafka.log.Log)
> [2015-09-18 10:28:49,376] INFO Recovering unflushed segment 1752741 in log pony-topic-85. (kafka.log.Log)
> [2015-09-18 10:28:49,389] INFO Completed load of log pony-topic-1 with log end offset 0 (kafka.log.Log)
> [2015-09-18 10:28:49,403] INFO Recovering unflushed segment 0 in log production-dropped-dtevents-topic-1. (kafka.log.Log)
> [2015-09-18 10:28:50,950] INFO Completed load of log pony-topic-73 with log end offset 2017625 (kafka.log.Log)
> [2015-09-18 10:28:50,967] INFO Completed load of log pony-topic-61 with log end offset 2020452 (kafka.log.Log)
> [2015-09-18 10:28:50,989] INFO Completed load of log pony-topic-85 with log end offset 2019928 (kafka.log.Log)
> [2015-09-18 10:28:50,996] INFO Completed load of log pony-topic-13 with log end offset 2018860 (kafka.log.Log)
> [2015-09-18 10:28:53,222] INFO Completed load of log production-dropped-dtevents-topic-1 with log end offset 893236 (kafka.log.Log)
> [2015-09-18 10:28:53,227] ERROR There was an error in one of the threads during logs loading: java.lang.IllegalArgumentException: requirement failed: Corrupt index found, index file (/mnt/21e57274-0ba1-41e0-9ea4-befef32ec62c/data/kafka/pony-topic-1/00000000000001753742.index) has non-zero size but the last offset is 1753742 and the base offset is 1753742 (kafka.log.LogManager)
> [2015-09-18 10:28:53,230] FATAL [Kafka Server 6], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
> java.lang.IllegalArgumentException: requirement failed: Corrupt index found, index file (/mnt/21e57274-0ba1-41e0-9ea4-befef32ec62c/data/kafka/pony-topic-1/00000000000001753742.index) has non-zero size but the last offset is 1753742 and the base offset is 1753742
> at scala.Predef$.require(Predef.scala:219)
> at kafka.log.OffsetIndex.sanityCheck(OffsetIndex.scala:352)
> at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:184)
> at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:183)
> at scala.collection.Iterator$class.foreach(Iterator.scala:750)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1202)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at kafka.log.Log.loadSegments(Log.scala:183)
> at kafka.log.Log.<init>(Log.scala:67)
> at kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$7$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:142)
> at kafka.utils.Utils$$anon$1.run(Utils.scala:54)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> [2015-09-18 10:28:53,234] INFO [Kafka Server 6], shutting down (kafka.server.KafkaServer)
> [2015-09-18 10:28:53,240] INFO [Kafka Server 6], shut down completed (kafka.server.KafkaServer)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)