[jira] [Resolved] (GOBBLIN-1057) Remove unnecessary RPCs in distcp-ng

["Hung Tran (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GOBBLIN-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hung Tran resolved GOBBLIN-1057.
--------------------------------
    Fix Version/s: 0.15.0
       Resolution: Fixed

Issue resolved by pull request #2897
[https://github.com/apache/incubator-gobblin/pull/2897]

> Remove unnecessary RPCs in distcp-ng
> ------------------------------------
>
>                 Key: GOBBLIN-1057
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1057
>             Project: Apache Gobblin
>          Issue Type: Improvement
>            Reporter: Hung Tran
>            Priority: Major
>             Fix For: 0.15.0
>
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> There are some per-file FileSystem RPCs being invoked in Gobblin distcp-ng.
> This results in a long file discovery phase that can be hours for a few thousand files.
> The RPCs that can be removed are:
> getFileChecksum() - the value doesn't appear to be used.
> getFileStatus() - this is called to get the modification time in ModTimeDataFileVersionStrategy.getVersion(). The modification time is already available from listStatus(), so use that value.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)