[jira] [Resolved] (HDFS-12943) Consistent Reads from Standby Node

["Konstantin Shvachko (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko resolved HDFS-12943.
----------------------------------------
    Fix Version/s: 2.10.0
                   3.2.2
                   3.1.4
                   3.3.0
     Hadoop Flags: Reviewed
     Release Note: 
Observer is a new type of NameNodes in addition to Active and Standby in HA settings. Observer Node maintains a replica of the namespace same as a Standby Node. It additionally allows execution of clients read requests.
To ensure read-after-write consistency within a single client, a state ID is introduced in RPC headers. The Observer responds to the client request only after its own state has caught up with the client’s state ID, which it previously received from the Active NameNode.
Clients can explicitly invoke a new client protocol call msync(), which ensures that subsequent reads by this client from an Observer are consistent.
A new client-side ObserverReadProxyProvider is introduced to provide automatic switching between Active and Observer NameNodes for submitting respectively write and read requests.
       Resolution: Fixed

Closing this as Fixed. The feature has been tested, back-ported down to 2.10 and released. Few remaining subtasks are being addressed as usual issues.
Added release notes. Please review if I missed anything.

_Thank you everybody for contributing to this effort._

> Consistent Reads from Standby Node
> ----------------------------------
>
>                 Key: HDFS-12943
>                 URL: https://issues.apache.org/jira/browse/HDFS-12943
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: hdfs
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>            Priority: Major
>             Fix For: 3.3.0, 3.1.4, 3.2.2, 2.10.0
>
>         Attachments: ConsistentReadsFromStandbyNode.pdf, ConsistentReadsFromStandbyNode.pdf, HDFS-12943-001.patch, HDFS-12943-002.patch, HDFS-12943-003.patch, HDFS-12943-004.patch, TestPlan-ConsistentReadsFromStandbyNode.pdf
>
>
> StandbyNode in HDFS is a replica of the active NameNode. The states of the NameNodes are coordinated via the journal. It is natural to consider StandbyNode as a read-only replica. As with any replicated distributed system the problem of stale reads should be resolved. Our main goal is to provide reads from standby in a consistent way in order to enable a wide range of existing applications running on top of HDFS.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org