[jira] [Commented] (HDFS-17147) RBF: RouterRpcServer getListing become extremely slow when the children of the dir are mounted in the same ns.

["Yuanbo Liu (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751980#comment-17751980 ] 

Yuanbo Liu commented on HDFS-17147:
-----------------------------------

cc [~inigoiri]  [~zhangshuyan] [~ayushsaxena] ,

> RBF: RouterRpcServer getListing become extremely slow when the children of the dir are mounted in the same ns.
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-17147
>                 URL: https://issues.apache.org/jira/browse/HDFS-17147
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: rbf
>            Reporter: Yuanbo Liu
>            Priority: Major
>
> Suppose we mount table as below:
>  
> {code:java}
> /dir -> ns0 ->  /target/dir
> /dir/child1 -> ns0 -> /target/dir/child1
> /dir/child2 -> ns0 -> /target/dir/child2
> ......
> /dir/child200 -> ns0 -> /target/dir/child200
> {code}
>  
>  
> when listing /dir with RBF, it's getting extremely slow as getListing has two parts:
> 1. list all children of  /target/dir
> 2. append the rest 200 mount points to the result.
>  
> The second part invoke getFileInfo concurrently to make sure mount points are accessed under rightful permission. But in this case, the first part includes the result of the second part, and there is no need to append second part repeatly.
>  
>  
>  
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org