You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "Sudarshan Vasudevan (Jira)" <ji...@apache.org> on 2020/10/07 19:05:00 UTC
[jira] [Updated] (GOBBLIN-742) Implement a FileSystem based
JobConfigurationManager
[ https://issues.apache.org/jira/browse/GOBBLIN-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sudarshan Vasudevan updated GOBBLIN-742:
----------------------------------------
Fix Version/s: 0.15.0
> Implement a FileSystem based JobConfigurationManager
> ----------------------------------------------------
>
> Key: GOBBLIN-742
> URL: https://issues.apache.org/jira/browse/GOBBLIN-742
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-cluster
> Affects Versions: 0.15.0
> Reporter: Sudarshan Vasudevan
> Assignee: Hung Tran
> Priority: Major
> Fix For: 0.15.0, 0.16.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> This task aims to implement a FileSystem based JobConfigurationManager. The manager has an underlying SpecConsumer which reads specs from a config specified path on HDFS and persists them to a JobCatalog. On successful consumption of the spec by the manager , it is deleted from the original path. So on the next fetch, only new specs will be picked up by the JobConfigurationManager.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)