You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Daryn Sharp (JIRA)" <ji...@apache.org> on 2017/07/14 15:05:00 UTC
[jira] [Updated] (HDFS-12140) Remove BPOfferService lock contention
to get block pool id
[ https://issues.apache.org/jira/browse/HDFS-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-12140:
-------------------------------
Attachment: HDFS-12140.trunk.patch
Simply sets/clears a volatile for the bp id when the ns info is updated. Retains the blocking behavior prior to setting the ns info, but lockless access thereafter.
Will post branch-2 patch shortly. Minor change needed for the absent "quiet" boolean.
> Remove BPOfferService lock contention to get block pool id
> ----------------------------------------------------------
>
> Key: HDFS-12140
> URL: https://issues.apache.org/jira/browse/HDFS-12140
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.8.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HDFS-12140.trunk.patch
>
>
> The block pool id is protected by a lock in {{BPOfferService}}. This creates excessive contention especially for xceivers threads attempting to queue IBRs and heartbeat processing. When the latter is delayed due to excessive FSDataset lock contention, it causes pipelines to collapse.
> Accessing the block pool id should be lockless after registration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org