You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Chenzhao Guo (JIRA)" <ji...@apache.org> on 2019/06/21 03:43:00 UTC
[jira] [Created] (HDFS-14592) Support NIO transferTo semantics in
HDFS
Chenzhao Guo created HDFS-14592:
-----------------------------------
Summary: Support NIO transferTo semantics in HDFS
Key: HDFS-14592
URL: https://issues.apache.org/jira/browse/HDFS-14592
Project: Hadoop HDFS
Issue Type: New Feature
Reporter: Chenzhao Guo
I'm currently developing a Spark shuffle manager based on HDFS. I need to merge some spill files on HDFS to one, or rearrange some HDFS files.
An API similar to NIO transferTo, which bypasses memory will be more efficient than manually reading and writing bytes(the method I'm using at present).
So can HDFS implements something like NIO transferTo? Making path.transferTo(pathDestination) possible?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org