[jira] [Created] (APLO-359) Cannot shut down when stuck in startup

["Adam Hooper (JIRA)" <ji...@apache.org>]

Adam Hooper created APLO-359:
--------------------------------

             Summary: Cannot shut down when stuck in startup
                 Key: APLO-359
                 URL: https://issues.apache.org/jira/browse/APLO-359
             Project: ActiveMQ Apollo
          Issue Type: Bug
          Components: apollo-broker, apollo-cli
    Affects Versions: 1.6
         Environment: Ubuntu 14.04
            Reporter: Adam Hooper


Steps to reproduce:

1. In Terminal A, start ApolloMQ via apollo-cli.
2. In Terminal B, do the same thing with the same options from the same working directory. Notice the log message: `WARN  | DB operation failed. (entering recovery mode): java.io.IOException: File 'message-broker/data/lock' could not be locked.`
3. In Terminal C, kill (SIGTERM) the ApolloMQ you created in Terminal B.

Expected results: the process from Terminal B dies.

Actual results: the process from Terminal B stays open, and only a SIGKILL can kill it. That's because the shutdown hook waits for the broker to stop, but the broker won't stop because it never started.

Apollo's shutdown hook is a huge sysadmin headache for us. It hurts us in two places:

A) On our cloud server, Upstart sends a SIGTERM and waits before sending a final SIGKILL. That means it takes an excessive amount of time for us to restart Apollo: we need to "stop; sleep 6; start" or we can't know the restart will succeed. Sure, we can solve this with extra sysadminning that sends SIGKILL judiciously; but if we're supposed to SIGKILL Apollo, then why is there a shutdown hook in the first place?

B) We distribute a download-and-run parent program that invokes and monitors Apollo. This is a cross-platform Java program, so it can only send a SIGTERM. If Apollo doesn't shut down from a SIGTERM, it stays running forever, and it blocks new Apollo processes from starting.



--
This message was sent by Atlassian JIRA
(v6.2#6252)