[GitHub] [accumulo] milleruntime commented on a diff in pull request #2990: Block use of fate in Manager until fate is ready

[GitBox <gi...@apache.org>]

milleruntime commented on code in PR #2990:
URL: https://github.com/apache/accumulo/pull/2990#discussion_r988327743


##########
server/manager/src/main/java/org/apache/accumulo/manager/Manager.java:
##########
@@ -225,6 +230,40 @@ public boolean stillManager() {
     return getManagerState() != ManagerState.STOP;
   }
 
+  /**
+   * Retrieve the Fate object, blocking until it is ready. This could cause problems if Fate
+   * operations are attempted to be used prior to the Manager being ready for them. If these
+   * operations are triggered by a client side request from a tserver or client, it should be safe
+   * to wait to handle those until Fate is ready, but if it occurs during an upgrade, or some other
+   * time in the Manager before Fate is started, that may result in a deadlock and will need to be
+   * fixed.
+   *
+   * @return the Fate object, only after the fate components are running and ready
+   */
+  Fate<Manager> fate() {
+    try {
+      // block up to 30 seconds until it's ready; if it's still not ready, introduce some logging
+      if (!fateReadyLatch.await(30, TimeUnit.SECONDS)) {
+        String msgPrefix = "Unexpected use of fate in thread " + Thread.currentThread().getName()
+            + " at time " + System.currentTimeMillis();
+        // include stack trace so we know where it's coming from, in case we need to troubleshoot it
+        log.warn("{} blocked until fate starts", msgPrefix,
+            new IllegalStateException("Attempted fate action before manager finished starting up; "
+                + "if this doesn't make progress, please report it as a bug to the developers"));
+        int minutes = 0;
+        while (!fateReadyLatch.await(5, TimeUnit.MINUTES)) {
+          minutes += 5;
+          log.warn("{} still blocked after {} minutes; this is getting weird", msgPrefix, minutes);
+        }
+        log.debug("{} no longer blocked", msgPrefix);
+      }

Review Comment:
   Maybe. We don't want to kill the thread though cause they were just started in the Manager as part of the startup.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org