You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Rakesh Radhakrishnan (Jira)" <ji...@apache.org> on 2020/01/02 07:01:00 UTC

[jira] [Updated] (HDFS-14740) Recover data blocks from persistent memory read cache during datanode restarts

     [ https://issues.apache.org/jira/browse/HDFS-14740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rakesh Radhakrishnan updated HDFS-14740:
----------------------------------------
    Fix Version/s: 3.2.2
                   3.1.4
                   3.3.0
       Resolution: Fixed
           Status: Resolved  (was: Patch Available)

I have committed latest patch to the respective branches - trunk, branch-3.2 and branch-3.1.

Thanks [~PhiloHe] and [~Rui Mo] for the contribution.



> Recover data blocks from persistent memory read cache during datanode restarts
> ------------------------------------------------------------------------------
>
>                 Key: HDFS-14740
>                 URL: https://issues.apache.org/jira/browse/HDFS-14740
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: caching, datanode
>            Reporter: Feilong He
>            Assignee: Feilong He
>            Priority: Major
>             Fix For: 3.3.0, 3.1.4, 3.2.2
>
>         Attachments: HDFS-14740-branch-3.1-000.patch, HDFS-14740-branch-3.1-001.patch, HDFS-14740-branch-3.2-000.patch, HDFS-14740-branch-3.2-001.patch, HDFS-14740.000.patch, HDFS-14740.001.patch, HDFS-14740.002.patch, HDFS-14740.003.patch, HDFS-14740.004.patch, HDFS-14740.005.patch, HDFS-14740.006.patch, HDFS-14740.007.patch, HDFS-14740.008.patch, HDFS-14740.009.patch, HDFS_Persistent_Read-Cache_Design-v1.pdf, HDFS_Persistent_Read-Cache_Test-v1.1.pdf, HDFS_Persistent_Read-Cache_Test-v1.pdf, HDFS_Persistent_Read-Cache_Test-v2.pdf
>
>
> In HDFS-13762, persistent memory (PM) is enabled in HDFS centralized cache management. Even though PM can persist cache data, for simplifying the initial implementation, the previous cache data will be cleaned up during DataNode restarts. Here, we are proposing to improve HDFS PM cache by taking advantage of PM's data persistence characteristic, i.e., recovering the status for cached data, if any, when DataNode restarts, thus, cache warm up time can be saved for user.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org