[jira] [Commented] (HDFS-9783) DataNode deadlocks after running hdfs mover

["Vinayakumar B (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-9783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15139135#comment-15139135 ] 

Vinayakumar B commented on HDFS-9783:
-------------------------------------

This issue has been already fixed in HDFS-9661 which is available in branch-2.7 ( 2.7.3)
And the stacktrace shown is not part of hadoop-2.6 code. {{FsDataSetImpl#moveBlockAcrossStorage(..)}} not present in 2.6 code.

So IMO this would be a duplicate.

> DataNode deadlocks after running hdfs mover
> -------------------------------------------
>
>                 Key: HDFS-9783
>                 URL: https://issues.apache.org/jira/browse/HDFS-9783
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>    Affects Versions: 2.6.0
>            Reporter: David Watzke
>         Attachments: datanode.stack.txt
>
>
> We're running CDH 5.4.4 (with Hadoop 2.6.0). Recently we've added 800T of ARCHIVE storage, marked some data (16T * 2 repl. factor) as "COLD" and ran the hdfs mover to enforce the storage policy.
> After a few minutes, the mover hangs because one or more datanodes hang as well. Please check out the deadlock revealed by jstack. Also, here's what appeared in DN log:
> 2016-02-09 15:58:15,676 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Not able to copy block 1157230686 to /5.45.60.142:40144 because threads quota is exceeded.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)