[jira] [Updated] (HUDI-8165) Implements LRU eviction strategy for in-memory map of ExternalSpillableMap

["Ethan Guo (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ethan Guo updated HUDI-8165:
----------------------------
    Fix Version/s: 1.1.0
                       (was: 1.0.0)

> Implements LRU eviction strategy for in-memory map of ExternalSpillableMap
> --------------------------------------------------------------------------
>
>                 Key: HUDI-8165
>                 URL: https://issues.apache.org/jira/browse/HUDI-8165
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: core
>            Reporter: Danny Chen
>            Assignee: Danny Chen
>            Priority: Critical
>             Fix For: 1.1.0
>
>
> The current impl for the map does not use any eviction strategy, it just keeps the records that put first in it. A more proper way is we introduce a LRU eviction strategy, the in-memory cache can keep the keys that are accessed more frequent.
> Imagine there is hot key that comes recent but spilled to disk, the LRU cache can ensure it is kept in memory which avoid unnecessary disk SE/DE overhead.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)