[jira] [Created] (HIVE-24171) Support HDFS reads from observer NameNodes

["Aihua Xu (Jira)" <ji...@apache.org>]

Aihua Xu created HIVE-24171:
-------------------------------

             Summary: Support HDFS reads from observer NameNodes
                 Key: HIVE-24171
                 URL: https://issues.apache.org/jira/browse/HIVE-24171
             Project: Hive
          Issue Type: New Feature
          Components: Hive
    Affects Versions: 3.0.0
            Reporter: Aihua Xu
            Assignee: Aihua Xu


HDFS-12943 introduces the consistent reads from observer NameNodes which can boost the read performance and reduces the overloads on active NameNodes.

To take advantage of this feature, the clients are required to make a msync() call after writing the files or before reading the files since observer NameNodes could have the stale data for a small window. 

Hive needs to make msync() call to HDFS in some places, e.g., 1) after generating the plan files - map.xml and reduce.xml so they can get used later by executors; 2) after the intermediate files are generated so they can get used by later stages or HS2. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)