You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Sudarshan Vasudevan (Jira)" <ji...@apache.org> on 2021/01/06 21:47:00 UTC

[jira] [Created] (GOBBLIN-1352) Ensure Helix workflows are cleaned up on cluster start up in centralized mode

Sudarshan Vasudevan created GOBBLIN-1352:
--------------------------------------------

             Summary: Ensure Helix workflows are cleaned up on cluster start up in centralized mode
                 Key: GOBBLIN-1352
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1352
             Project: Apache Gobblin
          Issue Type: Bug
          Components: gobblin-cluster
    Affects Versions: 0.16.0
            Reporter: Sudarshan Vasudevan
            Assignee: Hung Tran
             Fix For: 0.16.0


As part of the changes introduced in GOBBLIN-1319, the Helix workflow names were modified to include a special prefix. However, this prefix is intended for use in distributed mode. As a consequence of this change, the workflows are no longer being cleaned up on cluster start up leading to a leak in Znodes. Further, since the workflows are left in the START state, the Yarn autoscaling manager requests more containers than needed.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)