[jira] [Updated] (IGNITE-4337) Introduce persistence interface to allow build reliable persistence plugins

["Alexey Goncharuk (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IGNITE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexey Goncharuk updated IGNITE-4337:
-------------------------------------
    Description: 
With page memory interface introduced, it may be possible to build a persistence layer around this architecture. I think we should move the PageMemory interface itself to {{org.apache.ignite.plugin.storage}} package and introduce the following interface to allow other components to log it's activity in crash-resistant way:

{code}
/**
 *
 */
public interface IgniteWriteAheadLogManager extends GridCacheSharedManager {
    /**
     * @return {@code true} If we have to always write full pages.
     */
    public boolean isAlwaysWriteFullPages();

    /**
     * @return {@code true} if WAL will perform fair syncs on fsync call.
     */
    public boolean isFullSync();

    /**
     * Resumes logging after start. When WAL manager is started, it will skip logging any updates until this
     * method is called to avoid logging changes induced by the state restore procedure.
     */
    public void resumeLogging(WALPointer lastWrittenPtr) throws IgniteCheckedException;

    /**
     * Appends the given log entry to the write-ahead log.
     *
     * @param entry entry to log.
     * @return WALPointer that may be passed to {@link #fsync(WALPointer)} method to make sure the record is
     *      written to the log.
     * @throws IgniteCheckedException If failed to construct log entry.
     * @throws StorageException If IO error occurred while writing log entry.
     */
    public WALPointer log(WALRecord entry) throws IgniteCheckedException, StorageException;

    /**
     * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to
     * the underlying storage.
     *
     * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record.
     * @throws IgniteCheckedException If
     * @throws StorageException
     */
    public void fsync(WALPointer ptr) throws IgniteCheckedException, StorageException;

    /**
     * Invoke this method to iterate over the written log entries.
     *
     * @param start Optional WAL pointer from which to start iteration.
     * @return Records iterator.
     * @throws IgniteException If failed to start iteration.
     * @throws StorageException If IO error occurred while reading WAL entries.
     */
    public WALIterator replay(WALPointer start) throws IgniteCheckedException, StorageException;

    /**
     * Gives a hint to WAL manager to clear entries logged before the given pointer. Some entries before the
     * the given pointer will be kept because there is a configurable WAL history size. Those entries may be used
     * for partial partition rebalancing.
     *
     * @param ptr Pointer for which it is safe to clear the log.
     * @return Number of deleted WAL segments.
     */
    public int truncate(WALPointer ptr);
}
{code}

  was:
If page memory interface is introduced, it may be possible to build a persistence layer around this architecture. I think we should add some form of persistence logging to allow us build a crash-resistant system in future.

Something like
{code}
public interface IgnitePersistenceLogManager extends GridCacheSharedManager {
    /**
     * @return {@code true} If we have to always write full pages.
     */
    public boolean isAlwaysWriteFullPages();

    /**
     * @return {@code true} if persistence manager will perform fair syncs on fsync call.
     */
    public boolean isFullSync();

    /**
     * Resumes logging after start. When persistence manager is started, it will skip logging any updates until this
     * method is called to avoid logging changes induced by the state restore procedure.
     */
    public void resumeLogging(LogPointer lastWrittenPtr) throws IgniteCheckedException;

    /**
     * Appends the given log entry to the write-ahead log.
     *
     * @param entry entry to log.
     * @return LogPointer that may be passed to {@link #fsync(LogPointer)} method to make sure the record is
     *      written to the log.
     * @throws IgniteCheckedException If failed to construct log entry.
     * @throws StorageException If IO error occurred while writing log entry.
     */
    public LogPointer log(LogRecord entry) throws IgniteCheckedException, StorageException;

    /**
     * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to
     * the underlying storage.
     *
     * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record.
     * @throws IgniteCheckedException If
     * @throws StorageException
     */
    public void fsync(LogPointer ptr) throws IgniteCheckedException, StorageException;

    /**
     * Invoke this method to iterate over the written log entries.
     *
     * @param start Optional log pointer from which to start iteration.
     * @return Records iterator.
     * @throws IgniteException If failed to start iteration.
     * @throws StorageException If IO error occurred while reading log entries.
     */
    public LogIterator replay(LogPointer start) throws IgniteCheckedException, StorageException;

    /**
     * Gives a hint to persistence manager to clear entries logged before the given pointer. Some entries before the
     * the given pointer will be kept because there is a configurable log history size. Those entries may be used
     * for partial partition rebalancing.
     *
     * @param ptr Pointer for which it is safe to clear the log.
     * @return Number of deleted log segments.
     */
    public int truncate(LogPointer ptr);
}
{code}


> Introduce persistence interface to allow build reliable persistence plugins
> ---------------------------------------------------------------------------
>
>                 Key: IGNITE-4337
>                 URL: https://issues.apache.org/jira/browse/IGNITE-4337
>             Project: Ignite
>          Issue Type: Sub-task
>          Components: general
>            Reporter: Alexey Goncharuk
>             Fix For: 2.0
>
>
> With page memory interface introduced, it may be possible to build a persistence layer around this architecture. I think we should move the PageMemory interface itself to {{org.apache.ignite.plugin.storage}} package and introduce the following interface to allow other components to log it's activity in crash-resistant way:
> {code}
> /**
>  *
>  */
> public interface IgniteWriteAheadLogManager extends GridCacheSharedManager {
>     /**
>      * @return {@code true} If we have to always write full pages.
>      */
>     public boolean isAlwaysWriteFullPages();
>     /**
>      * @return {@code true} if WAL will perform fair syncs on fsync call.
>      */
>     public boolean isFullSync();
>     /**
>      * Resumes logging after start. When WAL manager is started, it will skip logging any updates until this
>      * method is called to avoid logging changes induced by the state restore procedure.
>      */
>     public void resumeLogging(WALPointer lastWrittenPtr) throws IgniteCheckedException;
>     /**
>      * Appends the given log entry to the write-ahead log.
>      *
>      * @param entry entry to log.
>      * @return WALPointer that may be passed to {@link #fsync(WALPointer)} method to make sure the record is
>      *      written to the log.
>      * @throws IgniteCheckedException If failed to construct log entry.
>      * @throws StorageException If IO error occurred while writing log entry.
>      */
>     public WALPointer log(WALRecord entry) throws IgniteCheckedException, StorageException;
>     /**
>      * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to
>      * the underlying storage.
>      *
>      * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record.
>      * @throws IgniteCheckedException If
>      * @throws StorageException
>      */
>     public void fsync(WALPointer ptr) throws IgniteCheckedException, StorageException;
>     /**
>      * Invoke this method to iterate over the written log entries.
>      *
>      * @param start Optional WAL pointer from which to start iteration.
>      * @return Records iterator.
>      * @throws IgniteException If failed to start iteration.
>      * @throws StorageException If IO error occurred while reading WAL entries.
>      */
>     public WALIterator replay(WALPointer start) throws IgniteCheckedException, StorageException;
>     /**
>      * Gives a hint to WAL manager to clear entries logged before the given pointer. Some entries before the
>      * the given pointer will be kept because there is a configurable WAL history size. Those entries may be used
>      * for partial partition rebalancing.
>      *
>      * @param ptr Pointer for which it is safe to clear the log.
>      * @return Number of deleted WAL segments.
>      */
>     public int truncate(WALPointer ptr);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)