[jira] [Created] (GOBBLIN-1772) Helix offline instance purging is not thread safe in the yarn service

["Matthew Ho (Jira)" <ji...@apache.org>]

Matthew Ho created GOBBLIN-1772:
-----------------------------------

             Summary: Helix offline instance purging is not thread safe in the yarn service
                 Key: GOBBLIN-1772
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1772
             Project: Apache Gobblin
          Issue Type: Bug
          Components: gobblin-cluster
            Reporter: Matthew Ho
            Assignee: Hung Tran


The application master will request containers while helix offline instance purging is happening.

 
Problem:
The helix offline instance purging is not thread safe. The desired behavior is for the yarn service to finish purging before requesting any new containers.

 

But currently, the yarn autoscaling manager is making calls to the yarnservice to request more containers. And this is bypassing the startup block in the startup method.

 

This is because the startup itself isn't synchronous. It's possible to call methods on this object before startup is complete.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)