You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Hung Tran (JIRA)" <ji...@apache.org> on 2019/06/06 23:41:00 UTC

[jira] [Created] (GOBBLIN-798) Cleanup workflows from Helix when the Gobblin application master starts

Hung Tran created GOBBLIN-798:
---------------------------------

             Summary: Cleanup workflows from Helix when the Gobblin application master starts
                 Key: GOBBLIN-798
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-798
             Project: Apache Gobblin
          Issue Type: Task
            Reporter: Hung Tran
            Assignee: Hung Tran


If the application master aborts a new one may be spawned by YARN. The second application master will resubmit the jobs. This results in duplicate jobs in Helix and multiple instances of the job may run, resulting in duplicate data.

The Gobblin application master should clean up all workflows on startup to avoid executing multiple instances of a job.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)