[jira] [Created] (HDFS-13923) Add a configuration to turn on/off observer reads

["Chao Sun (JIRA)" <ji...@apache.org>]

Chao Sun created HDFS-13923:
-------------------------------

             Summary: Add a configuration to turn on/off observer reads
                 Key: HDFS-13923
                 URL: https://issues.apache.org/jira/browse/HDFS-13923
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Chao Sun


In some situations having a config to turn on/off observer reads dynamically may be useful. For instance, some applications may be sensitive for data freshness and want to always reach directly to active NN. In a more complex scenario, services such as Presto may want to apply observer reads for different types of queries. In this case, simply change {{dfs.client.failover.proxy.provider.<nameservice>}} may not be enough, since with FileSystem cache (which is usually turned on) will ignore the change and still use the same FileSystem object.

Here I'm proposing to add a flag in {{HdfsClientConfigKeys}}, such as {{dfs.client.observer.reads.enabled}}, that can be used to dynamically turn on/off observer reads. The FileSystem cache key should also take account of this flag in its {{hashCode}} and {{equals}} impl, so that different FileSystem objects will be used depending on the flag.

 

cc [~shv], [~xkrogen], [~vagarychen], [~zero45] for discussion.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org