You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2018/02/12 15:30:02 UTC
[jira] [Updated] (HADOOP-13432) S3A: Consider using
TransferManager.download for copyToLocalFile
[ https://issues.apache.org/jira/browse/HADOOP-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-13432:
------------------------------------
Parent: HADOOP-15220 (was: HADOOP-14831)
> S3A: Consider using TransferManager.download for copyToLocalFile
> ----------------------------------------------------------------
>
> Key: HADOOP-13432
> URL: https://issues.apache.org/jira/browse/HADOOP-13432
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Rajesh Balamohan
> Priority: Minor
>
> Currently it relies on the default implementation of FileSystem. But it would be good to explore using TransferManager.download() (Ref: https://java.awsblog.com/post/Tx3Z7NO7C2TVLB/Parallelizing-Large-Downloads-for-Optimal-Speed for recent aws-sdk-java). When aws-sdk version is bumped it,it would automatically get the benefit of parallel download sas well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org