[jira] [Resolved] (HBASE-30407) Add persistence support for cache components

["Vladimir Rodionov (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-30407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vladimir Rodionov resolved HBASE-30407.
---------------------------------------
    Resolution: Implemented

> Add persistence support for cache components
> --------------------------------------------
>
>                 Key: HBASE-30407
>                 URL: https://issues.apache.org/jira/browse/HBASE-30407
>             Project: HBase
>          Issue Type: New Feature
>          Components: BlockCache
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0-alpha-1
>
>
> h2. Description
> The pluggable block cache architecture currently has no common mechanism for persisting and
> restoring cache state across process restarts.
> Persistence may be required by multiple stateful components of the cache system, including
> {{CacheEngine}}, {{CacheTopology}}, and {{CachePlacementAdmissionPolicy}}. For example, a cache
> engine may persist cached data or index metadata, while a topology or placement/admission policy
> may maintain state that should survive a restart.
> Introduce a common persistence capability that can be implemented by cache components that need
> to save and restore their state.
> Persistence should operate on streams rather than files or paths. This keeps storage management
> outside of individual cache components and allows the caller to decide where persisted state is
> stored.
> Components should be constructed normally from the current HBase configuration before their
> persisted state is restored. Persistence is therefore responsible for restoring runtime state,
> not for constructing or configuring cache components.
> h2. Proposed Changes
> * Introduce a common persistence interface for cache components, e.g.
>   {{PersistentCacheComponent}}.
> * Allow {{CacheEngine}}, {{CacheTopology}}, and {{CachePlacementAdmissionPolicy}} implementations
>   to support this capability.
> * Provide a capability method such as {{supportsPersistence()}}.
> * Provide {{restore(InputStream)}} for restoring persisted component state.
> * Provide {{save(OutputStream)}} for saving component state.
> * Streams are supplied and owned by the caller. Cache components must not close them.
> * Persistence format is implementation-specific.
> * A component restores only state owned by that component. It must not recursively persist or
>   restore child components.
> * Add tests for persistence capability detection and state save/restore.
> h2. Proposed API
> {code:java}
> @InterfaceAudience.Private
> public interface PersistentCacheComponent {
>   /**
>    * Returns whether this component supports persistence.
>    *
>    * @return {@code true} if persistence is supported
>    */
>   default boolean supportsPersistence() {
>     return false;
>   }
>   /**
>    * Restores persisted state into this component.
>    * <p>
>    * The component must already be initialized. Persisted state is restored subject to the
>    * component's current configuration. This method does not close the supplied stream.
>    *
>    * @param input input stream containing persisted state
>    * @throws IOException if persisted state cannot be restored
>    * @throws UnsupportedOperationException if persistence is not supported
>    */
>   default void restore(InputStream input) throws IOException {
>     throw new UnsupportedOperationException("Persistence is not supported");
>   }
>   /**
>    * Saves this component's state to the specified output stream.
>    * <p>
>    * This method does not close the supplied stream.
>    *
>    * @param output output stream to which state is written
>    * @throws IOException if state cannot be saved
>    * @throws UnsupportedOperationException if persistence is not supported
>    */
>   default void save(OutputStream output) throws IOException {
>     throw new UnsupportedOperationException("Persistence is not supported");
>   }
> }
> {code}
> h2. Persistence Model
> Cache component construction remains configuration-driven.
> On startup, the cache hierarchy, engines, topology, and placement/admission policy are first
> constructed using the current HBase configuration. Persisted state is then restored into those
> existing instances.
> This provides a clear separation between configuration and runtime state:
> * Configuration defines which cache components exist and how they are configured.
> * Cache factories construct the components.
> * Persistence restores runtime state from a previous process instance.
> Persisted state therefore does not need to contain enough information to construct a cache
> component.
> Current configuration takes precedence over persisted configuration-related state. For example,
> if a cache engine was previously configured with a larger capacity and is restarted with a
> smaller capacity, restored state must be subject to the current cache configuration and
> constraints.
> h2. Component Ownership
> Persistence is local to each component.
> For example:
> * A {{CacheEngine}} may persist cached blocks, indexes, or engine-specific eviction metadata.
> * A {{CacheTopology}} may persist topology-specific runtime state.
> * A {{CachePlacementAdmissionPolicy}} may persist frequency, admission, placement, or other
>   learned state.
> A component's {{save()}} and {{restore()}} methods operate only on state owned directly by that
> component. A topology must not implicitly persist its cache engines, and a cache engine must not
> persist topology or policy state.
> This allows higher-level cache infrastructure to coordinate persistence without coupling
> individual components to each other's persistence formats.
> h2. Stream Ownership
> {{save()}} and {{restore()}} operate on caller-provided streams.
> Cache components:
> * must not open persistence files directly;
> * must not depend on a particular filesystem or storage implementation;
> * must not close the supplied {{InputStream}} or {{OutputStream}};
> * may define their own versioned persistence format.
> This allows persisted cache state to be stored in local files, distributed filesystems, object
> stores, or other storage systems without exposing those concerns to cache implementations.
> h2. Acceptance Criteria
> * A common persistence capability is available to cache components.
> * Persistent components can save state to an {{OutputStream}}.
> * Persistent components can restore state from an {{InputStream}}.
> * Cache components are constructed from the current configuration before state is restored.
> * Restoring state does not construct or replace the cache component instance.
> * Current cache configuration remains authoritative when restoring persisted state.
> * Components persist only state they own.
> * Cache components do not close caller-provided streams.
> * Components that do not support persistence are unaffected.
> * Tests verify persistence capability detection and save/restore behavior.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)