You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Hung Tran (JIRA)" <ji...@apache.org> on 2019/06/06 23:42:00 UTC

[jira] [Updated] (GOBBLIN-798) Clean up workflows from Helix when the Gobblin application master starts

     [ https://issues.apache.org/jira/browse/GOBBLIN-798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hung Tran updated GOBBLIN-798:
------------------------------
    Summary: Clean up workflows from Helix when the Gobblin application master starts  (was: Cleanup workflows from Helix when the Gobblin application master starts)

> Clean up workflows from Helix when the Gobblin application master starts
> ------------------------------------------------------------------------
>
>                 Key: GOBBLIN-798
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-798
>             Project: Apache Gobblin
>          Issue Type: Task
>            Reporter: Hung Tran
>            Assignee: Hung Tran
>            Priority: Major
>
> If the application master aborts a new one may be spawned by YARN. The second application master will resubmit the jobs. This results in duplicate jobs in Helix and multiple instances of the job may run, resulting in duplicate data.
> The Gobblin application master should clean up all workflows on startup to avoid executing multiple instances of a job.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)