You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Sudarshan Vasudevan (JIRA)" <ji...@apache.org> on 2019/04/18 21:06:00 UTC

[jira] [Created] (GOBBLIN-742) Implement a FileSystem based JobConfigurationManager

Sudarshan Vasudevan created GOBBLIN-742:
-------------------------------------------

             Summary: Implement a FileSystem based JobConfigurationManager
                 Key: GOBBLIN-742
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-742
             Project: Apache Gobblin
          Issue Type: Improvement
          Components: gobblin-cluster
    Affects Versions: 0.15.0
            Reporter: Sudarshan Vasudevan
            Assignee: Hung Tran
             Fix For: 0.15.0


This task aims to implement a FileSystem based JobConfigurationManager. The manager has an underlying SpecConsumer which reads specs from a config specified path on HDFS and persists them to a JobCatalog. On successful consumption of the spec by the manager , it is deleted from the original path.  So on the next fetch, only new specs will be picked up by the JobConfigurationManager.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)