You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Arpit Agarwal (JIRA)" <ji...@apache.org> on 2018/05/08 22:19:00 UTC

[jira] [Created] (HADOOP-15451) Avoid fsync storm triggered by DiskChecker and handle disk full situation

Arpit Agarwal created HADOOP-15451:
--------------------------------------

             Summary: Avoid fsync storm triggered by DiskChecker and handle disk full situation
                 Key: HADOOP-15451
                 URL: https://issues.apache.org/jira/browse/HADOOP-15451
             Project: Hadoop Common
          Issue Type: Bug
            Reporter: Kihwal Lee
            Assignee: Arpit Agarwal


Fix disk checker issues reported by [~kihwal] in HADOOP-13738:
# When space is low, the os returns ENOSPC. Instead simply stop writing, the drive is marked bad and replication happens. This make cluster-wide space problem worse. If the number of "failed" drives exceeds the DFIP limit, the datanode shuts down.
# There are non-hdfs users of DiskChecker, who use it proactively, not just on failures. This was fine before, but now it incurs heavy I/O due to introduction of fsync() in the code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org