You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Erik Krogen (JIRA)" <ji...@apache.org> on 2018/06/20 01:03:00 UTC
[jira] [Created] (HDFS-13689) NameNodeRpcServer getEditsFromTxid
assumes it is run on active NameNode
Erik Krogen created HDFS-13689:
----------------------------------
Summary: NameNodeRpcServer getEditsFromTxid assumes it is run on active NameNode
Key: HDFS-13689
URL: https://issues.apache.org/jira/browse/HDFS-13689
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs, namenode
Reporter: Erik Krogen
Assignee: Erik Krogen
{{NameNodeRpcServer#getEditsFromTxid}} currently decides which transactions are able to be served, i.e. which transactions are durable, using the following logic:
{code}
long syncTxid = log.getSyncTxId();
// If we haven't synced anything yet, we can only read finalized
// segments since we can't reliably determine which txns in in-progress
// segments have actually been committed (e.g. written to a quorum of JNs).
// If we have synced txns, we can definitely read up to syncTxid since
// syncTxid is only updated after a transaction is committed to all
// journals. (In-progress segments written by old writers are already
// discarded for us, so if we read any in-progress segments they are
// guaranteed to have been written by this NameNode.)
boolean readInProgress = syncTxid > 0;
{code}
This assumes that the NameNode serving this request is the current writer/active NameNode, which may not be true in the ObserverNode situation. Since {{selectInputStreams}} now has a {{onlyDurableTxns}} flag, which, if enabled, will only return durable/committed transactions, we can instead leverage this to provide the same functionality. We should utilize this to avoid consistency issues when serving this request from the ObserverNode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org