[jira] [Commented] (FALCON-287) Record lineage information in post processing

["Shaik Idris Ali (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FALCON-287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13920953#comment-13920953 ] 

Shaik Idris Ali commented on FALCON-287:
----------------------------------------

May be I missed the actual use-case of lineage, why do we need to persist it and who are the consumers of this information.

Secondly, instead of FalconPostProcessing storing this data on HDFS, which might further slowdown user workflow.
A service can subscribe to FALCON.ENTITY.TOPIC and record the details in HDFS or anywhere.
I think we discussed about this doing in ProcessSubscriberService.

Can we discuss this over a chat? 11 PM IST.


> Record lineage information in post processing
> ---------------------------------------------
>
>                 Key: FALCON-287
>                 URL: https://issues.apache.org/jira/browse/FALCON-287
>             Project: Falcon
>          Issue Type: Sub-task
>    Affects Versions: 0.5
>            Reporter: Venkatesh Seetharam
>            Assignee: Venkatesh Seetharam
>              Labels: lineage
>         Attachments: FALCON-287-v1.patch, FALCON-287-v2.patch, FALCON-287.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)