You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Sudarshan Vasudevan (Jira)" <ji...@apache.org> on 2020/10/07 18:51:00 UTC

[jira] [Updated] (GOBBLIN-1266) Refactor dataset lineage code to allow Lineage event emission in streaming mode

     [ https://issues.apache.org/jira/browse/GOBBLIN-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sudarshan Vasudevan updated GOBBLIN-1266:
-----------------------------------------
    Fix Version/s:     (was: 0.16.0)
                   0.15.0

> Refactor dataset lineage code to allow Lineage event emission in streaming mode 
> --------------------------------------------------------------------------------
>
>                 Key: GOBBLIN-1266
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1266
>             Project: Apache Gobblin
>          Issue Type: Improvement
>          Components: gobblin-core
>    Affects Versions: 0.15.0
>            Reporter: Sudarshan Vasudevan
>            Assignee: Abhishek Tiwari
>            Priority: Major
>             Fix For: 0.15.0
>
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Dataset Lineage emission in current Gobblin code is only available for batch mode of execution. For instance, the lineage events are emitted on dataset commit, which is never invoked for long-running tasks. This PR refactors lineage related code so that it is leverageable in streaming mode of execution as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)