[jira] [Resolved] (KAFKA-12712) KRaft: Missing controller.quorom.voters config not properly handled

["Colin McCabe (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Colin McCabe resolved KAFKA-12712.
----------------------------------
    Fix Version/s: 3.0.0
                       (was: 3.2.0)
         Assignee: Jason Gustafson
       Resolution: Fixed

> KRaft: Missing controller.quorom.voters config not properly handled
> -------------------------------------------------------------------
>
>                 Key: KAFKA-12712
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12712
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 2.8.0
>            Reporter: Magnus Edenhill
>            Assignee: Jason Gustafson
>            Priority: Major
>              Labels: kip-500
>             Fix For: 3.0.0
>
>
> When trying out KRaft in 2.8 I mispelled controller.quorum.voters as controller.quorum.voters, but the broker did not fail to start, nor did it print any warning.
>  
> Instead it raised this error:
>  
> {code:java}
> [2021-04-23 18:25:13,484] INFO Starting controller (kafka.server.ControllerServer)[2021-04-23 18:25:13,484] INFO Starting controller (kafka.server.ControllerServer)[2021-04-23 18:25:13,485] ERROR [kafka-raft-io-thread]: Error due to (kafka.raft.KafkaRaftManager$RaftIoThread)java.lang.IllegalArgumentException: bound must be positive at java.util.Random.nextInt(Random.java:388) at org.apache.kafka.raft.RequestManager.findReadyVoter(RequestManager.java:57) at org.apache.kafka.raft.KafkaRaftClient.maybeSendAnyVoterFetch(KafkaRaftClient.java:1778) at org.apache.kafka.raft.KafkaRaftClient.pollUnattachedAsObserver(KafkaRaftClient.java:2080) at org.apache.kafka.raft.KafkaRaftClient.pollUnattached(KafkaRaftClient.java:2061) at org.apache.kafka.raft.KafkaRaftClient.pollCurrentState(KafkaRaftClient.java:2096) at org.apache.kafka.raft.KafkaRaftClient.poll(KafkaRaftClient.java:2181) at kafka.raft.KafkaRaftManager$RaftIoThread.doWork(RaftManager.scala:53) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
> {code}
> which I guess eventually (1 minute later) lead to this error which terminated the broker:
> {code:java}
> [2021-04-23 18:26:14,435] ERROR [BrokerLifecycleManager id=2] Shutting down because we were unable to register with the controller quorum. (kafka.server.BrokerLifecycleManager)[2021-04-23 18:26:14,435] ERROR [BrokerLifecycleManager id=2] Shutting down because we were unable to register with the controller quorum. (kafka.server.BrokerLifecycleManager)[2021-04-23 18:26:14,436] INFO [BrokerLifecycleManager id=2] registrationTimeout: shutting down event queue. (org.apache.kafka.queue.KafkaEventQueue)[2021-04-23 18:26:14,437] INFO [BrokerLifecycleManager id=2] Transitioning from STARTING to SHUTTING_DOWN. (kafka.server.BrokerLifecycleManager)[2021-04-23 18:26:14,437] INFO [broker-2-to-controller-send-thread]: Shutting down (kafka.server.BrokerToControllerRequestThread)[2021-04-23 18:26:14,438] INFO [broker-2-to-controller-send-thread]: Stopped (kafka.server.BrokerToControllerRequestThread)[2021-04-23 18:26:14,438] INFO [broker-2-to-controller-send-thread]: Shutdown completed (kafka.server.BrokerToControllerRequestThread)[2021-04-23 18:26:14,441] ERROR [BrokerServer id=2] Fatal error during broker startup. Prepare to shutdown (kafka.server.BrokerServer)java.util.concurrent.CancellationException at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) at kafka.server.BrokerLifecycleManager$ShutdownEvent.run(BrokerLifecycleManager.scala:474) at org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:174) at java.lang.Thread.run(Thread.java:748)
> {code}
> But since the client listeners were made available prior to shutting down, the broker was deemed up and operational by the (naiive) monitoring tool.
> So..:
>  - Broker should fail on startup on invalid/unknown config properties. I understand this is tehcnically tricky, so at least a warning log should be printed.
>  - Perhaps not create client listeners before control plane is somewhat happy.
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)