[jira] [Commented] (KAFKA-742) Existing directories under the Kafka data directory without any data cause process to not start

["Jay Kreps (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565696#comment-13565696 ] 

Jay Kreps commented on KAFKA-742:
---------------------------------

Getting an error is not really bad since we assume we have control of everything under the log directory (it is obviously hard for us to distinguish a log directory from a non-log directory and silently ignoring could be worse than an error). So I think there are two problems here:
1. Are we enabling metrics logging by default? Are we creating the metrics dir even if metrics logging is not enabled. This needs to be sanity checked...
2. If there are bogus directories under the log directory I think the right thing to do is to give a better error message (something like "Found directory /x/y/z, 'z' is not in the form topic-partition").

 If you agree with those fixes I will take this on.
                
> Existing directories under the Kafka data directory without any data cause process to not start
> -----------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-742
>                 URL: https://issues.apache.org/jira/browse/KAFKA-742
>             Project: Kafka
>          Issue Type: Bug
>          Components: config
>    Affects Versions: 0.8
>            Reporter: Chris Curtin
>
> I incorrectly setup the configuration file to have the metrics go to /var/kafka/metrics while the logs were in /var/kafka. On startup I received the following error then the daemon exited:
> 30   [main] INFO  kafka.log.LogManager  - [Log Manager on Broker 0] Loading log 'metrics'
> 32   [main] FATAL kafka.server.KafkaServerStartable  - Fatal error during KafkaServerStable startup. Prepare to shutdown
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
>         at java.lang.String.substring(String.java:1937)
>         at kafka.log.LogManager.kafka$log$LogManager$$parseTopicPartitionName(LogManager.scala:335)
>         at kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$3.apply(LogManager.scala:112)
>         at kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$3.apply(LogManager.scala:109)
>         at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
>         at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)
>         at kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:109)
>         at kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:101)
>         at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
>         at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32)
>         at kafka.log.LogManager.loadLogs(LogManager.scala:101)
>         at kafka.log.LogManager.<init>(LogManager.scala:62)
>         at kafka.server.KafkaServer.startup(KafkaServer.scala:59)
>         at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
>         at kafka.Kafka$.main(Kafka.scala:46)
>         at kafka.Kafka.main(Kafka.scala)
> 34   [main] INFO  kafka.server.KafkaServer  - [Kafka Server 0], shutting down
> This was on a brand new cluster so no data or metrics logs existed yet.
> Moving the metrics to their own directory (not a child of the logs) allowed the daemon to start.
> Took a few minutes to figure out what was wrong.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira