You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Chenzhao Guo (JIRA)" <ji...@apache.org> on 2019/06/21 03:43:00 UTC

[jira] [Created] (HDFS-14592) Support NIO transferTo semantics in HDFS

Chenzhao Guo created HDFS-14592:
-----------------------------------

             Summary: Support NIO transferTo semantics in HDFS
                 Key: HDFS-14592
                 URL: https://issues.apache.org/jira/browse/HDFS-14592
             Project: Hadoop HDFS
          Issue Type: New Feature
            Reporter: Chenzhao Guo


I'm currently developing a Spark shuffle manager based on HDFS. I need to merge some spill files on HDFS to one, or rearrange some HDFS files.

An API similar to NIO transferTo, which bypasses memory will be more efficient than manually reading and writing bytes(the method I'm using at present).

So can HDFS implements something like NIO transferTo? Making path.transferTo(pathDestination) possible? 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org