You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Julio Ng (JIRA)" <ji...@apache.org> on 2019/05/14 00:21:00 UTC
[jira] [Created] (KAFKA-8362) LogCleaner gets stuck after partition
move between log directories
Julio Ng created KAFKA-8362:
-------------------------------
Summary: LogCleaner gets stuck after partition move between log directories
Key: KAFKA-8362
URL: https://issues.apache.org/jira/browse/KAFKA-8362
Project: Kafka
Issue Type: Bug
Components: log cleaner
Reporter: Julio Ng
When a partition is moved from one directory to another, their checkpoint entry in cleaner-offset-checkpoint file is not removed from the source directory.
As a consequence when we read the last firstDirtyOffset, we might get a stale value from the old checkpoint file.
Basically, we need clean up the entry from the check point file in the source directory when the move is completed
The current issue is that the code in LogCleanerManager:
{{def allCleanerCheckpoints: Map[TopicPartition, Long] = {}}
{{ inLock(lock) {}}
{{ checkpoints.values.flatMap(checkpoint => {}}
{{ try {}}
{{ checkpoint.read()}}
{{ } catch {}}
{{ case e: KafkaStorageException =>}}
{{ error(s"Failed to access checkpoint file ${checkpoint.file.getName} in dir ${checkpoint.file.getParentFile.getAbsolutePath}", e)}}
{{ Map.empty[TopicPartition, Long]}}
{{ }}}
{{ }).toMap}}
{{ }}}
{{}}}
collapses the offsets when multiple entries exist for the topicPartition
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)