You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Wei-Chiu Chuang (Jira)" <ji...@apache.org> on 2022/01/18 21:00:00 UTC
[jira] [Commented] (HDFS-16423) balancer should not get blocks on stale storages
[ https://issues.apache.org/jira/browse/HDFS-16423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17478200#comment-17478200 ]
Wei-Chiu Chuang commented on HDFS-16423:
----------------------------------------
Thanks for filing the bug report!
I just wonder if HDFS-12914 has anything to do with the symptom you described here and in HDFS-16420.
Right after a NameNode starts, all storages are regarded as stale initially. There are supposedly FBRs happening afterwards, but for large clusters there used to be timeouts that caused FBR to expire and some storages become stale for an extended period of time. This is no more a problem after HDFS-12914, so I wonder if HDFS-16423 and HDFS-16420 became less likely.
Nothing to discount the importance of these bug reports! This is great to find out.
> balancer should not get blocks on stale storages
> ------------------------------------------------
>
> Key: HDFS-16423
> URL: https://issues.apache.org/jira/browse/HDFS-16423
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: balancer & mover
> Reporter: qinyuren
> Assignee: qinyuren
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-01-13-17-18-32-409.png
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> We have met a problems as described in HDFS-16420
> We found that balancer copied a block multi times without deleting the source block if this block was placed in a stale storage. And resulting a block with many copies, but these redundant copies are not deleted until the storage become not stale.
>
> !image-2022-01-13-17-18-32-409.png|width=657,height=275!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org