You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Neal Richardson (Jira)" <ji...@apache.org> on 2020/09/11 18:12:00 UTC
[jira] [Resolved] (ARROW-9868) [C++] Provide utility for copying
files between filesystems
[ https://issues.apache.org/jira/browse/ARROW-9868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neal Richardson resolved ARROW-9868.
------------------------------------
Resolution: Fixed
Issue resolved by pull request 8101
[https://github.com/apache/arrow/pull/8101]
> [C++] Provide utility for copying files between filesystems
> -----------------------------------------------------------
>
> Key: ARROW-9868
> URL: https://issues.apache.org/jira/browse/ARROW-9868
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
> Reporter: Neal Richardson
> Assignee: Ben Kietzman
> Priority: Major
> Labels: filesystem, pull-request-available, s3
> Fix For: 2.0.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> {{CopyStream}} in arrow/filesystem/util_internal.h does this, but we should expose it, multithread it (can read in one thread while the other thread writes), and further see if there are filesystem-specific optimizations (e.g. S3 multipart uploading/downloading). We may also want a version that takes a FileSelector or vector of paths and parallelizes the operations on them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)