You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Sudarshan Vasudevan (Jira)" <ji...@apache.org> on 2020/09/19 19:22:00 UTC

[jira] [Created] (GOBBLIN-1266) Refactor dataset lineage code to allow Lineage event emission in streaming mode

Sudarshan Vasudevan created GOBBLIN-1266:
--------------------------------------------

             Summary: Refactor dataset lineage code to allow Lineage event emission in streaming mode 
                 Key: GOBBLIN-1266
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1266
             Project: Apache Gobblin
          Issue Type: Improvement
          Components: gobblin-core
    Affects Versions: 0.15.0
            Reporter: Sudarshan Vasudevan
            Assignee: Abhishek Tiwari
             Fix For: 0.16.0


Dataset Lineage emission in current Gobblin code is only available for batch mode of execution. For instance, the lineage events are emitted on dataset commit, which is never invoked for long-running tasks. This PR refactors lineage related code so that it is leverageable in streaming mode of execution as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)