[jira] [Commented] (FALCON-1480) Gather data transfer details of Hive DR

["Peeyush Bishnoi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FALCON-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906128#comment-14906128 ] 

Peeyush Bishnoi commented on FALCON-1480:
-----------------------------------------

Approach for this to create custom mapreduce counters for the Hive replication job. If distcp jobs are invoked from Hive replication map tasks, populate custom MR counters in each map task with required counters value obtained from the completed distcp jobs.
Once Hive replication job completed successfully, obtain and store aggregate custom counters from Hive replication job in a counter file. Once counter file created, details for populating the GraphDB is similar as I have done in FALCON-1102. 


> Gather data transfer details of Hive DR
> ---------------------------------------
>
>                 Key: FALCON-1480
>                 URL: https://issues.apache.org/jira/browse/FALCON-1480
>             Project: Falcon
>          Issue Type: Sub-task
>          Components: replication
>            Reporter: Peeyush Bishnoi
>            Assignee: Peeyush Bishnoi
>
> Hive DR invokes replication job to transfer Hive metadata and data from source to destination. It is required to capture the data transfer details like bytes transferred, files copied of replication job and then populate the the backend store from where UI can access.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)