[jira] [Created] (KAFKA-9970) Kafka 2.4.0 crashes under Windows when client is restarted

["SledgeHammer (Jira)" <ji...@apache.org>]

SledgeHammer created KAFKA-9970:
-----------------------------------

             Summary: Kafka 2.4.0 crashes under Windows when client is restarted
                 Key: KAFKA-9970
                 URL: https://issues.apache.org/jira/browse/KAFKA-9970
             Project: Kafka
          Issue Type: Bug
          Components: core
    Affects Versions: 2.4.0
            Reporter: SledgeHammer


Windows 10 x64 Pro

JDK 11.0.7

Zookeeper 3.6.0

Kafka 2.12-2.4.0

 

I have reproduced this scenario on multiple machines. I do dev on my Windows box, so have ZK and K running locally. On my work PC, I'll leave ZK & K running in command windows for "ever" at home I'll shutdown when I'm not doing dev.

In either case, Spring Boot client is continuously started and restarted. Intermittently Kafka will crash and corrupt the logs (I'm not able to capture the K crash exception since it closes), but upon restart, I get the exception below. NOTE: file is not in use since I can delete the logs directory and then restart. Client is both streams and classic queues.

 

[2020-05-07 13:38:27,782] ERROR Failed to clean up log for __consumer_offsets-20 in dir C:\PROGRA~1\kafka_2.12-2.4.0\logs due to IOException (kafka.server.LogDirFailureChannel)
java.nio.file.FileSystemException: C:\PROGRA~1\kafka_2.12-2.4.0\logs\__consumer_offsets-20\00000000000000000000.timeindex.cleaned -> C:\PROGRA~1\kafka_2.12-2.4.0\logs\__consumer_offsets-20\00000000000000000000.timeindex.swap: The process cannot access the file because it is being used by another process.

at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
 at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
 at java.base/sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:395)
 at java.base/sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:292)
 at java.base/java.nio.file.Files.move(Files.java:1421)
 at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:795)
 at kafka.log.AbstractIndex.renameTo(AbstractIndex.scala:209)
 at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:497)
 at kafka.log.Log.$anonfun$replaceSegments$4(Log.scala:2267)
 at kafka.log.Log.$anonfun$replaceSegments$4$adapted(Log.scala:2267)
 at scala.collection.immutable.List.foreach(List.scala:392)
 at kafka.log.Log.replaceSegments(Log.scala:2267)
 at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:604)
 at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:529)
 at kafka.log.Cleaner.$anonfun$doClean$6$adapted(LogCleaner.scala:528)
 at scala.collection.immutable.List.foreach(List.scala:392)
 at kafka.log.Cleaner.doClean(LogCleaner.scala:528)
 at kafka.log.Cleaner.clean(LogCleaner.scala:502)
 at kafka.log.LogCleaner$CleanerThread.cleanLog(LogCleaner.scala:371)
 at kafka.log.LogCleaner$CleanerThread.cleanFilthiestLog(LogCleaner.scala:344)
 at kafka.log.LogCleaner$CleanerThread.tryCleanFilthiestLog(LogCleaner.scala:324)
 at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:313)
 at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)