[GitHub] [druid] gianm commented on pull request #11962: Attempt to debug CI problems.

[GitBox <gi...@apache.org>]

gianm commented on pull request #11962:
URL: https://github.com/apache/druid/pull/11962#issuecomment-974684001


   These log messages showed up.
   
   ```
   2021-11-20T17:10:55,648 ERROR [kafka-scheduler-7] kafka.server.ReplicaManager - [ReplicaManager broker=0] Error while writing to highwatermark file in directory /tmp/junit3186742195405664411/junit1710273094024018589
   
   org.apache.kafka.common.errors.KafkaStorageException: Error while writing to checkpoint file /tmp/junit3186742195405664411/junit1710273094024018589/replication-offset-checkpoint
   
   Caused by: java.io.FileNotFoundException: /tmp/junit3186742195405664411/junit1710273094024018589/replication-offset-checkpoint.tmp (No such file or directory)
   	at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_292]
   	at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_292]
   	at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[?:1.8.0_292]
   	at java.io.FileOutputStream.<init>(FileOutputStream.java:162) ~[?:1.8.0_292]
   	at kafka.server.checkpoints.CheckpointFile.liftedTree1$1(CheckpointFile.scala:94) ~[kafka_2.13-3.0.0.jar:?]
   	at kafka.server.checkpoints.CheckpointFile.write(CheckpointFile.scala:92) ~[kafka_2.13-3.0.0.jar:?]
   	at kafka.server.checkpoints.OffsetCheckpointFile.write(OffsetCheckpointFile.scala:67) ~[kafka_2.13-3.0.0.jar:?]
   ```
   
   Also this one.
   
   ```
   2021-11-20T17:10:55,716 ERROR [LogDirFailureHandler] kafka.log.LogManager - Shutdown broker because all log dirs in /tmp/junit3186742195405664411/junit1710273094024018589 have failed
   ```
   
   The latter one is from this Kafka code, which look suspiciously like it's going to lead to a `System.exit` right after the log message.
   
   ```
       if (liveLogDirs.isEmpty) {
         fatal(s"Shutdown broker because none of the specified log dirs from ${dirs.mkString(", ")} can be created or validated")
         Exit.halt(1)
       }
   ```
   
   I think the test might be deleting the log dirs before shutting down the broker, leading the broker to shut _itself_ down, along with the entire test JVM. I'll try changing that and see what happens.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org