You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Martin Dickson (Jira)" <ji...@apache.org> on 2024/09/17 10:54:00 UTC

[jira] [Created] (KAFKA-17562) Failure detection for degraded brokers

Martin Dickson created KAFKA-17562:
--------------------------------------

             Summary: Failure detection for degraded brokers
                 Key: KAFKA-17562
                 URL: https://issues.apache.org/jira/browse/KAFKA-17562
             Project: Kafka
          Issue Type: Improvement
          Components: core, replication
            Reporter: Martin Dickson


Follow on from [this mailing list discussion|[https://lists.apache.org/thread/z8xn2dm1zm3clymhh60hf7rzgw286k8q].]

When a leader for a partition becomes degraded but does not fully fail it can remove all follower replicas from ISR. This can happen solely due to a problem with the leader (slow disk, degraded network, ...), and hence a single failure can make the partition unavailable for writes (assuming min.insync.replicas=2). If the leader then fully fails the partition goes offline, which introduces data loss risks during recovery.

The recovery options will improve substantially with KIP-966 (again assuming min.insync.replicas=2), but we there is still a gap around failure detection. In particular, KIP-966 alone doesn't help with the case when the broker is degraded but does not fully fail for a long period of time.

Currently Kafka failure detection is based on whether the broker can maintain its connection with the metadata quorum. The suggestion here is to consider more comprehensive failure detection, which could be handled by demoting leadership rather than fully fencing.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)