You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "Hung Tran (JIRA)" <ji...@apache.org> on 2019/04/16 07:01:00 UTC
[jira] [Updated] (GOBBLIN-739) Add a way to propagate the Azkaban
job config to Gobblin on YARN
[ https://issues.apache.org/jira/browse/GOBBLIN-739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hung Tran updated GOBBLIN-739:
------------------------------
Summary: Add a way to propagate the Azkaban job config to Gobblin on YARN (was: Add a way to propagate the Azkaban config to Gobblin on YARN)
> Add a way to propagate the Azkaban job config to Gobblin on YARN
> ----------------------------------------------------------------
>
> Key: GOBBLIN-739
> URL: https://issues.apache.org/jira/browse/GOBBLIN-739
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Hung Tran
> Assignee: Hung Tran
> Priority: Major
>
> The AzkabanGobblinYarnAppLauncher can be used to launch a Gobblin application master on YARN, which then loads configuration from an application.conf file. Currently, the application.conf is pre-generated and packaged with the Azkaban job zip. This results in duplication of config between the Azkaban job properties and the application.conf file. It also doesn't allow user overrides in the Azkaban UI to be propagated to the app master and containers.
> A config should be added to specify an output path to write the Azkaban job config to in HOCON format. The gobblin yarn config such as gobblin.yarn.app.master.files.local and gobblin.yarn.container.files.local can be set to point to the output file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)