You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Yang Yun (Jira)" <ji...@apache.org> on 2019/12/09 03:32:00 UTC
[jira] [Created] (HDFS-15039) Cache meta file length of
FinalizedReplica to reduce call File.length()
Yang Yun created HDFS-15039:
-------------------------------
Summary: Cache meta file length of FinalizedReplica to reduce call File.length()
Key: HDFS-15039
URL: https://issues.apache.org/jira/browse/HDFS-15039
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Reporter: Yang Yun
Assignee: Yang Yun
When use ReplicaCachingGetSpaceUsed to get the volume space used. It will call File.length() for every meta file of replica. That add more disk IO, we found the slow log as below. For finalized replica, the size of meta file is not changed, i think we can cache the value.
{code:java}
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed: Refresh dfs used, bpid: BP-898717543-10.75.1.240-1519386995727 replicas size: 1166 dfsUsed: 72227113183 on volume: DS-3add8d62-d69a-4f5a-a29f-b7bbb400af2e duration: 17206ms{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org