You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Shilun Fan (Jira)" <ji...@apache.org> on 2024/01/04 08:12:00 UTC
[jira] [Commented] (HDFS-15714) HDFS Provided Storage Read/Write Mount Support On-the-fly
[ https://issues.apache.org/jira/browse/HDFS-15714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802622#comment-17802622 ]
Shilun Fan commented on HDFS-15714:
-----------------------------------
Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a blocker. Retarget 3.5.0.
> HDFS Provided Storage Read/Write Mount Support On-the-fly
> ---------------------------------------------------------
>
> Key: HDFS-15714
> URL: https://issues.apache.org/jira/browse/HDFS-15714
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, namenode
> Affects Versions: 3.4.0
> Reporter: Feilong He
> Assignee: Feilong He
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-15714-01.patch, HDFS-15714-02.patch, HDFS_Provided_Storage_Design-V1.pdf, HDFS_Provided_Storage_Performance-V1.pdf
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> HDFS Provided Storage (PS) is a feature to tier HDFS over other file systems. In HDFS-9806, PROVIDED storage type was introduced to HDFS. Through configuring external storage with PROVIDED tag for DataNode, user can enable application to access data stored externally from HDFS side. However, there are two issues need to be addressed. Firstly, mounting external storage on-the-fly, namely dynamic mount, is lacking. It is necessary to get it supported to flexibly combine HDFS with an external storage at runtime. Secondly, PS write is not supported by current HDFS. But in real applications, it is common to transfer data bi-directionally for read/write between HDFS and external storage.
> Through this JIRA, we are presenting our work for PS write support and dynamic mount support for both read & write. Please note in the community several JIRAs have been filed for these topics. Our work is based on these previous community work, with new design & implementation to support called writeBack mount and enable admin to add any mount on-the-fly. We appreciate those folks in the community for their great contribution! See their pending JIRAs: HDFS-14805 & HDFS-12090.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org