[jira] [Created] (KAFKA-7569) Kafka doesnt appear to cleanup dangling partitions

["Dao Quang Minh (JIRA)" <ji...@apache.org>]

Dao Quang Minh created KAFKA-7569:
-------------------------------------

             Summary: Kafka doesnt appear to cleanup dangling partitions
                 Key: KAFKA-7569
                 URL: https://issues.apache.org/jira/browse/KAFKA-7569
             Project: Kafka
          Issue Type: Bug
    Affects Versions: 1.0.0
            Reporter: Dao Quang Minh


In our current cluster running kafka 1.0.0, we recently observed that Kafka doesnt cleanup dangling partitions ( i.e. partion data on disk, but partition is not assigned to the current broker anymore ).

For example of the dangling partition data, we have:

{code}
total 26G
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:19 00000000352433304663.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:16 00000000352414164340.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:24 00000000352466972892.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:23 00000000352457368236.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:17 00000000352423709566.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:21 00000000352447702369.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:20 00000000352442921890.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:22 00000000352452551548.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:17 00000000352418945305.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:18 00000000352428477361.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:15 00000000352409416538.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:24 00000000352462192103.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:20 00000000352438136012.log
-rw-r--r-- 1 kafka kafka 1.8G Aug 6 17:43 00000000352471757311.log
-rw-r--r-- 1 kafka kafka 10M Oct 16 21:44 00000000352471757311.index
-rw-r--r-- 1 kafka kafka 10M Oct 16 21:44 00000000352471757311.timeindex
drwxr-xr-x 2 kafka kafka 4.0K Oct 8 15:27 .
drwxr-xr-x 49 kafka kafka 4.0K Oct 30 11:21 ..
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352414164340.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352423709566.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352433304663.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352447702369.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352457368236.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352466972892.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352409416538.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352418945305.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352428477361.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352438136012.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352442921890.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352452551548.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352462192103.timeindex
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352414164340.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352423709566.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352433304663.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352447702369.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352457368236.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352466972892.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352409416538.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352418945305.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352428477361.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352438136012.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352442921890.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352452551548.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352462192103.index
-rw-r--r-- 1 kafka kafka 20 Aug 6 16:23 leader-epoch-checkpoint
-rw-r--r-- 1 kafka kafka 10 Aug 6 16:24 00000000352466972892.snapshot
-rw-r--r-- 1 kafka kafka 10 Aug 6 16:24 00000000352471757311.snapshot
-rw-r--r-- 1 kafka kafka 10 Oct 8 15:27 00000000352476186724.snapshot
{code}

I'm unsure how we ended up in this situation as partition data should be marked as removed and eventually remove when it's not assigned to the broker anymore. But in this edge case, should Kafka detect that automatically when it loads the partition and re-mark it as to be deleted again ?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)