[jira] [Commented] (KAFKA-1074) Reassign partitions should delete the old replicas from disk

["Jun Rao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13826647#comment-13826647 ] 

Jun Rao commented on KAFKA-1074:
--------------------------------

Removing the old replica log from the disk itself is simple. What we need to make sure is that all potential outstanding operations on the deleted log are handled properly. In particular, we don't want any potential IOException due to the missing log causes the broker to halt.

1. All read operations are ok since we already handle unexpected exceptions in KafkaApi.

2. Writing to the log by the producer request, the replica fetcher or the log flusher: We need to make sure that after the log is deleted, no more writes/flushes will be attempted on the log. This can be achieved by:
2.1 For producer requests, the delete partition operation will synchronize on the leaderAndIsrUpdate lock.
2.2 For replica fetcher, this is already handled since the fetcher is removed before the log is deleted.
2.3 For log flusher, the flush and the delete will now synchronize on a delete lock.

> Reassign partitions should delete the old replicas from disk
> ------------------------------------------------------------
>
>                 Key: KAFKA-1074
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1074
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.8
>            Reporter: Jun Rao
>             Fix For: 0.8.1
>
>         Attachments: KAFKA-1074.patch
>
>
> Currently, after reassigning replicas to other brokers, the old replicas are not removed from disk and have to be deleted manually.



--
This message was sent by Atlassian JIRA
(v6.1#6144)