[jira] [Commented] (KAFKA-13334) ERROR Failed to clean up log for __consumer_offsets

["Maksim Zinal (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481628#comment-17481628 ] 

Maksim Zinal commented on KAFKA-13334:
--------------------------------------

Duplicating my comment from KAFKA-2170 here.

I've created a quick patch which seems to fix the issue on my system and on the Kafka version I use (2.8.0).

I believe that the reason of lock failures is the use of RandomAccessFile Java API to open the files in some cases, while in other cases FileChannel.open() is used instead. When opening files with RandomAccessFile under Windows, FILE_SHARE_DELETE flag is not set, which leads to "access denied" errors when trying to rename or delete the open files. FileChannel.open() sets the FILE_SHARE_DELETE by default, as I checked on JDK 8 and 11.

Here's the link to the branch based on tag 2.8.0: [https://github.com/zinal/kafka/tree/2.8.0_KAFKA-1194|[https://github.com/zinal/kafka/tree/2.8.0_KAFKA-1194].]

Here are the exact changes implemented: [https://github.com/zinal/kafka/compare/2.8.0...zinal:2.8.0_KAFKA-1194] (plus jcenter and grgit stuff needed to run the build).

> ERROR Failed to clean up log for __consumer_offsets
> ---------------------------------------------------
>
>                 Key: KAFKA-13334
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13334
>             Project: Kafka
>          Issue Type: Bug
>          Components: log cleaner
>    Affects Versions: 2.8.1
>         Environment: Windows Service application and Windows 10 operating system
>            Reporter: Udaya Kumar
>            Priority: Blocker
>         Attachments: server.log.2021-09-23-17
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> [2021-09-23 17:10:29,126] ERROR Failed to clean up log for __consumer_offsets-38 in dir C:\Kafka\server_logData due to IOException (kafka.server.LogDirFailureChannel)[2021-09-23 17:10:29,126] ERROR Failed to clean up log for __consumer_offsets-38 in dir C:\Kafka\server_logData due to IOException (kafka.server.LogDirFailureChannel)java.nio.file.FileSystemException: C:\Kafka\server_logData\__consumer_offsets-38\00000000000000000000.timeindex.cleaned -> C:\Kafka\server_logData\__consumer_offsets-38\00000000000000000000.timeindex.swap:The process cannot access the file because it is being used by another process at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92) at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103) at java.base/sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:401) at java.base/sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:293)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)