You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-issues@hadoop.apache.org by "zhangduo (JIRA)" <ji...@apache.org> on 2015/05/09 01:08:01 UTC

[jira] [Commented] (HDFS-5270) Use thread pools in the datenode daemons

    [ https://issues.apache.org/jira/browse/HDFS-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535829#comment-14535829 ] 

zhangduo commented on HDFS-5270:
--------------------------------

Datanode will interrupt the thread when stopping pipeline.This is not a safe operation when the thread is in a thread pool, maybe the problem is here. I'm currently working on a new patch that stopping pipeline by close peer, and waiting for writer quit by a Future. Hope I can finish in a few days.
Thanks. [~busbey] 

> Use thread pools in the datenode daemons
> ----------------------------------------
>
>                 Key: HDFS-5270
>                 URL: https://issues.apache.org/jira/browse/HDFS-5270
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>              Labels: BB2015-05-TBR
>         Attachments: HDFS-5270.000.patch, HDFS-5270.2.patch, TestConcurrentAccess.java
>
>
> The current implementation of the datanode creates a thread when a new request comes in. This incurs high overheads for the creation / destruction of threads, making the datanode unstable under high concurrent loads.
> This JIRA proposes to use a thread pool to reduce the overheads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)