[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11315: Enhance Minion task management

["Jackie-Jiang (via GitHub)" <gi...@apache.org>]

Jackie-Jiang commented on code in PR #11315:
URL: https://github.com/apache/pinot/pull/11315#discussion_r1290771548


##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotHelixTaskResourceManager.java:
##########
@@ -118,25 +117,19 @@ public synchronized Set<String> getTaskTypes() {
    *
    * @param taskType Task type
    */
-  public void ensureTaskQueueExists(String taskType) {
+  public synchronized void ensureTaskQueueExists(String taskType) {
     String helixJobQueueName = getHelixJobQueueName(taskType);
     WorkflowConfig workflowConfig = _taskDriver.getWorkflowConfig(helixJobQueueName);
-
     if (workflowConfig == null) {
       // Task queue does not exist
       LOGGER.info("Creating task queue: {} for task type: {}", helixJobQueueName, taskType);
 
       // Set full parallelism
       // Don't allow overlap job assignment so that we can control number of concurrent tasks per instance
-      JobQueue jobQueue = new JobQueue.Builder(helixJobQueueName)
-          .setWorkflowConfig(new WorkflowConfig.Builder().setParallelJobs(Integer.MAX_VALUE).build()).build();
+      JobQueue jobQueue = new JobQueue.Builder(helixJobQueueName).setWorkflowConfig(
+          new WorkflowConfig.Builder().setParallelJobs(Integer.MAX_VALUE).build()).build();
       _taskDriver.createQueue(jobQueue);
     }
-
-    // Wait until task queue context shows up
-    while (_taskDriver.getWorkflowContext(helixJobQueueName) == null) {

Review Comment:
   This was added in #1894 as a workaround to a Helix bug. Since we upgraded to latest Helix version, this workaround is no longer needed. Other changes were already removed, but this one was not removed because we need to fix the task status API as well



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org