[jira] [Commented] (HDFS-16059) dfsadmin -listOpenFiles -blockingDecommission can miss some files

["Akira Ajisaka (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17359872#comment-17359872 ] 

Akira Ajisaka commented on HDFS-16059:
--------------------------------------

The regression test result:
{quote}
[ERROR] Tests run: 10, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.864 s <<< FAILURE! - in org.apache.hadoop.hdfs.server.namenode.TestFSNamesystem
[ERROR] testGetFilesBlockingDecomInternal(org.apache.hadoop.hdfs.server.namenode.TestFSNamesystem)  Time elapsed: 0.305 s  <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<4>
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:633)
	at org.apache.hadoop.hdfs.server.namenode.TestFSNamesystem.testGetFilesBlockingDecomInternal(TestFSNamesystem.java:372) {quote}

> dfsadmin -listOpenFiles -blockingDecommission can miss some files
> -----------------------------------------------------------------
>
>                 Key: HDFS-16059
>                 URL: https://issues.apache.org/jira/browse/HDFS-16059
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: dfsadmin
>            Reporter: Akira Ajisaka
>            Priority: Major
>         Attachments: HDFS-16059-regression-test.patch
>
>
> While reviewing HDFS-13671, I found "dfsadmin -listOpenFiles -blockingDecommission" can drop some files.
> [https://github.com/apache/hadoop/pull/3065#discussion_r647396463]
> {quote}If the DataNodes have the following open files and we want to list all the open files:
> DN1: [1001, 1002, 1003, ... , 2000]
>  DN2: [1, 2, 3, ... , 1000]
> At first getFilesBlockingDecom(0, "/") is called and it returns [1001, 1002, ... , 2000] because it reached max size (=1000), and next getFilesBlockingDecom(2000, "/") is called because the last inode Id of the previous result is 2000. That way the open files of DN2 is missed
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org