[jira] [Created] (HDFS-13529) Fix default trash policy emptier trigger time correctly

["He Xiaoqiao (JIRA)" <ji...@apache.org>]

He Xiaoqiao created HDFS-13529:
----------------------------------

             Summary: Fix default trash policy emptier trigger time correctly
                 Key: HDFS-13529
                 URL: https://issues.apache.org/jira/browse/HDFS-13529
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: namenode
    Affects Versions: 2.7.6, 3.1.0, 3.2.0, 2.9.2, 2.8.5
            Reporter: He Xiaoqiao


Trash emptier is designed to auto trigger at UTC 00:00, however I am confused all the time that it usually triggers at a few minutes even half a hour after UTC 00:00 actually in our production cluster.
The main reason is default policy emptier thread sleep more time than as expect, since it does not consider the delete operation time cost itself. especially for a large cluster, auto trash cleaner may cost dozens of minutes.
The right way is that gets current time {{now}} before calculate {{end}} time.
{code:java}
      long now = Time.now();
      while (true) {
        end = ceiling(now, emptierInterval);
        try {                                     // sleep for interval
          Thread.sleep(end - now);
        } catch (InterruptedException e) {
          break;                                  // exit on interrupt
        }

        try {
          now = Time.now();
          ...... // delete trash checkpoint
        } catch (Exception e) {
          LOG.warn("RuntimeException during Trash.Emptier.run(): ", e); 
        }
      }
{code}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org