[jira] [Created] (HDFS-8964) Provide max TxId when validating in-progress edit log files

["Zhe Zhang (JIRA)" <ji...@apache.org>]

Zhe Zhang created HDFS-8964:
-------------------------------

             Summary: Provide max TxId when validating in-progress edit log files
                 Key: HDFS-8964
                 URL: https://issues.apache.org/jira/browse/HDFS-8964
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: journal-node, namenode
    Affects Versions: 2.7.1
            Reporter: Zhe Zhang
            Assignee: Zhe Zhang


NN/JN validates in-progress edit log files in multiple scenarios, via {{EditLogFile#validateLog}}. The method scans through the edit log file to find the last transaction ID.

However, an in-progress edit log file could be actively written to, which creates a race condition and causes incorrect data to be read (and later we attempt to interpret the data as ops).

Currently {{validateLog}} is used in 3 places:
# NN {{getEditsFromTxid}}
# JN {{getEditLogManifest}}
# NN/JN {{recoverUnfinalizedSegments}}

In the first two scenarios we should provide a maximum TxId to validate in the in-progress file. The 3rd scenario won't cause a race condition because only non-current in-progress edit log files are validated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)