[jira] [Commented] (HBASE-20952) Re-visit the WAL API

["Zach York (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568906#comment-16568906 ] 

Zach York commented on HBASE-20952:
-----------------------------------

So taking a very rough stab at this based on a few thoughts, but at the most basic level, what functionality do we need from the WAL?

 

Absolutely Necessary:

append/put/write/etc

getEditsForRegion(Region) - This let's the implementation handle the mess - WALSplitter, multi-wal, etc

Probably some sort of delete of edits before a threshold

 

Nice to have:

onRegionFlush() - This would be kinda like a coprocessor hook which could add implementation specific logic (rolling file logs, adding some sort of lastFlushed functionality, or any of the things that [~stack] was talking about in his comment)

 

In general, I would like to start from what we truly need, not what we currently have :) . I don't want the interface polluted by implementation specific methods. I'm sure I'm over simplifying things, but I wanted to get the conversation started.

> Re-visit the WAL API
> --------------------
>
>                 Key: HBASE-20952
>                 URL: https://issues.apache.org/jira/browse/HBASE-20952
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>            Reporter: Josh Elser
>            Priority: Major
>
> Take a step back from the current WAL implementations and think about what an HBase WAL API should look like. What are the primitive calls that we require to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We should also have a mind for what is happening in the Ratis LogService (but the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and backup&restore. Replication has the use-case for "tail"'ing the WAL which we should provide via our new API. B&R doesn't do anything fancy (IIRC). We should make sure all consumers are generally going to be OK with the API we create.
> The API may be "OK" (or OK in a part). We need to also consider other methods which were "bolted" on such as {{AbstractFSWAL}} and {{WALFileLengthProvider}}. Other corners of "WAL use" (like the {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability annotations are chosen.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)