You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/03/19 13:53:01 UTC

[GitHub] [kafka] ijuma opened a new pull request #10361: KAFKA-9189: Use MetadataCache instead of ZK during controlled shutdown

ijuma opened a new pull request #10361:
URL: https://github.com/apache/kafka/pull/10361


   This avoids hanging during shutdown if ZK is unavailable. We could change ZK
   calls to have a timeout, but I think this approach is better.
   
   The downside is that the metadata cache may be slightly out of date, but we will
   retry as per the controlled shutdown configuration. If this broker is partitioned
   away from the Controller and is not receiving metadata updates, then we want
   to shutdown asap anyway.
   
   I added a test that timed out without this change and included a couple of clean-ups
   in `ServerShutdownTest`:
   * Remove `testCleanShutdownWithDeleteTopicEnabled`, which is redundant
   since delete topics is enabled by default.
   * Remove redundant method arguments
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] ijuma merged pull request #10361: KAFKA-9189: Use MetadataCache instead of ZK during controlled shutdown to avoid hang

Posted by GitBox <gi...@apache.org>.
ijuma merged pull request #10361:
URL: https://github.com/apache/kafka/pull/10361


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] dajac commented on a change in pull request #10361: KAFKA-9189: Use MetadataCache instead of ZK during controlled shutdown

Posted by GitBox <gi...@apache.org>.
dajac commented on a change in pull request #10361:
URL: https://github.com/apache/kafka/pull/10361#discussion_r597709157



##########
File path: core/src/main/scala/kafka/server/KafkaServer.scala
##########
@@ -549,20 +548,17 @@ class KafkaServer(
       try {
 
         var remainingRetries = retries
-        var prevController: Broker = null
+        var prevController: MetadataBroker = null
         var ioException = false
 
         while (!shutdownSucceeded && remainingRetries > 0) {
           remainingRetries = remainingRetries - 1
 
           // 1. Find the controller and establish a connection to it.
-
-          // Get the current controller info. This is to ensure we use the most recent info to issue the
-          // controlled shutdown request.
           // If the controller id or the broker registration are missing, we sleep and retry (if there are remaining retries)
-          zkClient.getControllerId match {
+          metadataCache.getControllerId match {

Review comment:
       Should we update the `No controller registered in ZooKeeper` log below and say that there is no controller present in the metadata cache instead?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org