[GitHub] [druid] AmatyaAvadhanula commented on a diff in pull request #13223: Resume streaming tasks on Overlord switch

[GitBox <gi...@apache.org>]

AmatyaAvadhanula commented on code in PR #13223:
URL: https://github.com/apache/druid/pull/13223#discussion_r998912112


##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java:
##########
@@ -1950,6 +1950,45 @@ public Boolean apply(SeekableStreamIndexTaskRunner.Status status)
     // make sure the checkpoints are consistent with each other and with the metadata store
 
     verifyAndMergeCheckpoints(taskGroupsToVerify.values());
+
+    // A pause from the previous Overlord's supervisor, immediately before leader change
+    // can lead to tasks being in a state where they are active but do not read.
+    // If this is the first run, resume all existing active tasks to be safe
+    if (getState().isFirstRunOnly()) {
+      Map<String, ListenableFuture<Boolean>> activeTaskToResumeFutureMap = new HashMap<>();
+      for (TaskGroup taskGroup : activelyReadingTaskGroups.values()) {
+        for (String taskId : taskGroup.tasks.keySet()) {
+          activeTaskToResumeFutureMap.put(taskId, taskClient.resumeAsync(taskId));
+        }
+      }
+
+      for (Map.Entry<String, ListenableFuture<Boolean>> entry : activeTaskToResumeFutureMap.entrySet()) {
+        String taskId = entry.getKey();
+        ListenableFuture<Boolean> future = entry.getValue();
+        future.addListener(
+            new Runnable()
+            {
+              @Override
+              public void run()
+              {
+                try {
+                  if (entry.getValue().get()) {

Review Comment:
   Yes. Also, PUBLISHING tasks are not resumed. Only READING / PAUSED tasks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org