You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Daryn Sharp (JIRA)" <ji...@apache.org> on 2017/07/13 20:05:00 UTC

[jira] [Updated] (HDFS-12137) DN dataset lock should be fair

     [ https://issues.apache.org/jira/browse/HDFS-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daryn Sharp updated HDFS-12137:
-------------------------------
    Attachment: HDFS-12137.branch-2.patch
                HDFS-12137.trunk.patch

Passes a fair lock to instrumented lock instead of allowing it to implicitly create an unfair lock.

Only difference in trunk/branch-2 is context of import.

> DN dataset lock should be fair
> ------------------------------
>
>                 Key: HDFS-12137
>                 URL: https://issues.apache.org/jira/browse/HDFS-12137
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 2.8.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
>         Attachments: HDFS-12137.branch-2.patch, HDFS-12137.trunk.patch
>
>
> The dataset lock is very highly contended.  The unfair nature can be especially harmful to the heartbeat handling.  Under high loads, partially expose by HDFS-12136 introducing disk i/o within the lock, the heartbeat handling thread may process commands so slowly due to the contention that the node becomes stale or falsely declared dead.  The unfair lock is not helping and appears to be causing frequent starvation under load.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org