You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/06/26 08:46:00 UTC
[jira] [Resolved] (HADOOP-13432) S3A: Consider using
TransferManager.download for copyToLocalFile
[ https://issues.apache.org/jira/browse/HADOOP-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran resolved HADOOP-13432.
-------------------------------------
Resolution: Won't Fix
We've had enough trouble with xfer manager we don't want to commit to it more than needed
> S3A: Consider using TransferManager.download for copyToLocalFile
> ----------------------------------------------------------------
>
> Key: HADOOP-13432
> URL: https://issues.apache.org/jira/browse/HADOOP-13432
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Rajesh Balamohan
> Priority: Minor
>
> Currently it relies on the default implementation of FileSystem. But it would be good to explore using TransferManager.download() (Ref: https://java.awsblog.com/post/Tx3Z7NO7C2TVLB/Parallelizing-Large-Downloads-for-Optimal-Speed for recent aws-sdk-java). When aws-sdk version is bumped it,it would automatically get the benefit of parallel download sas well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org