[jira] [Commented] (KAFKA-768) broker should exit if hitting exceptions durin startup

["Guozhang Wang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277430#comment-14277430 ] 

Guozhang Wang commented on KAFKA-768:
-------------------------------------

We encountered a general deaklock issue with System.exit(), where a startup / shutdown aware container wrapping KafkaServerStartable that has shutdown hook executed by another thread is blocked waiting for the main thread in startup() to set the startup_finished flag while the main thread is blocked on waiting for the shutdown hook thread to join.

So I would like to propose a bit different solution:

1. Replace System.exit(1) in startup with the shutdown() call.
2. In KafkaServer.shutdown, move

{code}
        brokerState.newState(NotRunning)
        shutdownLatch.countDown()
        startupComplete.set(false)
{code}

to the final block.

3. Remove the System.exit(1) in shutdown() call.

This will resolve this issue but as well works with a startup / shutdown aware container that has a shutdown hook depending on the startup logic to be complete.

> broker should exit if hitting exceptions durin startup
> ------------------------------------------------------
>
>                 Key: KAFKA-768
>                 URL: https://issues.apache.org/jira/browse/KAFKA-768
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.8.0
>            Reporter: Jun Rao
>            Assignee: Jun Rao
>            Priority: Blocker
>              Labels: p1
>             Fix For: 0.8.0
>
>         Attachments: kafka-768.patch
>
>
> A broker hit the following exception, but didn't exit.
> 2013/02/20 01:54:21.341 FATAL [KafkaServerStartable] [main] [kafka] []  Fatal error during KafkaServerStable startup. Prepare to shutdown
> kafka.common.KafkaException: Failed to create data directory /export/content/kafka/i001_caches
>         at kafka.log.LogManager$$anonfun$createAndValidateLogDirs$2.apply(LogManager.scala:77)
>         at kafka.log.LogManager$$anonfun$createAndValidateLogDirs$2.apply(LogManager.scala:72)
>         at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
>         at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32)
>         at kafka.log.LogManager.createAndValidateLogDirs(LogManager.scala:72)
>         at kafka.log.LogManager.<init>(LogManager.scala:60)
>         at kafka.server.KafkaServer.startup(KafkaServer.scala:59)
>         at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)