[GitHub] [pulsar] kaushik-develop commented on a change in pull request #12178: Support for draining workers

[GitBox <gi...@apache.org>]

kaushik-develop commented on a change in pull request #12178:
URL: https://github.com/apache/pulsar/pull/12178#discussion_r717748796



##########
File path: pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/SchedulerManager.java
##########
@@ -415,6 +633,34 @@ private void compactAssignmentTopic() {
         }
     }
 
+    protected int updateWorkerDrainMap() {
+        long startTime = System.nanoTime();
+        int numRemovedWorkerIds = 0;
+
+        if (drainOpStatusMap.size() > 0) {

Review comment:
       I didn't quite understand the question: "Is there a reason we should synchronize this with drain operation?"
   
   updateWorkerDrainMap() does a periodic cleanup of stale information about drained workers. We need to do the operation some time after the drain has finished, when the drained worker is removed from the cluster. Since there is currently no hook (that I know of) into the SchedulerManager when a worker is added to, or removed from the cluster, the cleanup is done through a periodic poll (updateWorkerDrainMap). 
   
   The drain operation adds a record into the concurrent map [drainOpStatusMap] when a worker is drained. An implicit assumption in the system is that the drained worker will be removed from the system, soon, by an external orchestrator. When the drained worker is seen to be removed from the system, the drainOpStatusMap is cleaned up of stale information in the updateWorkerDrainMap() code.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org