You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Ye Ni (Jira)" <ji...@apache.org> on 2020/06/17 20:41:00 UTC
[jira] [Created] (HDFS-15417) Lazy get the datanode report for
federation WebHDFS operations
Ye Ni created HDFS-15417:
----------------------------
Summary: Lazy get the datanode report for federation WebHDFS operations
Key: HDFS-15417
URL: https://issues.apache.org/jira/browse/HDFS-15417
Project: Hadoop HDFS
Issue Type: Improvement
Components: federation, rbf, webhdfs
Reporter: Ye Ni
*Why*
For WebHDFS CREATE, OPEN, APPEND and GETFILECHECKSUM operations, router or namenode needs to get the datanodes where the block is located, then redirect the request to one of the datanodes.
However, this chooseDatanode action in router is much slower than namenode, which directly affects the WebHDFS operations above.
For namenode WebHDFS, it normally takes tens of milliseconds, while router always takes more than 2 seconds.
*How*
Only get the datanode report when necessary in router. It is a very expense operation where all the time is spent on.
This is only need when we want to exclude some datanodes or find a random datanode for CREATE.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org