You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Guangming Zhang <Gu...@arm.com> on 2017/11/14 09:00:48 UTC

HDFS unit test failure in AArch64. TestDirectoryScanner.testThrottling: Throttle is too permissive

Hadoop Version:  3.1.0-SNAPSHOT
Environment: ARMv8 AArch64, Ubuntu16.04
Description:  Hi,  When I ran the HDFS unit test and got a failure in TestDirectoryScanner.java test case :
TestDirectoryScanner.testThrottling:624 Throttle is too permissive
detail:
                Running org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
                Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 227.046 sec <<< FAILURE! - in org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
                testThrottling(org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner)  Time elapsed: 198.014 sec  <<< FAILURE!
                java.lang.AssertionError: Throttle is too permissive
                                                at org.junit.Assert.fail(Assert.java:88)
                                                at org.junit.Assert.assertTrue(Assert.java:41)
                                                at org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:624)

And below is the failure part of source code TestDirectoryScanner.java:
                      ...........
      while ((retries > 0) && ((ratio < 7f) || (ratio > 10f))) {
        scanner = new DirectoryScanner(dataNode, fds, conf);
        ratio = runThrottleTest(blocks);
        retries -= 1;
      }

      // Waiting should be about 9x running.
      LOG.info("RATIO: " + ratio);
      assertTrue("Throttle is too restrictive", ratio <= 10f);
      assertTrue("Throttle is too permissive", ratio >= 7f);
    ............
    private float runThrottleTest(int blocks) throws IOException {
      scanner.setRetainDiffs(true);
      scan(blocks, 0, 0, 0, 0, 0);
      scanner.shutdown();
      assertFalse(scanner.getRunStatus());
      return (float)scanner.timeWaitingMs.get() / scanner.timeRunningMs.get();
    }
  .............

The ratio in my test is 6.0578866, which is smaller than 7f in the code. So the code thrown out an assertTrue failure.
My questions are:
1. Why the ratio was set between 7f and 10f, is it a empirical value?
       2. The ratio is smaller than 7f in AArch64 platform, is this value within normal range?

Could anyone help? Thanks a lot.

Best Regards
Guangming Zhang
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.