You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Brahma Reddy Battula (Jira)" <ji...@apache.org> on 2020/04/11 18:15:04 UTC
[jira] [Updated] (HDFS-11464) Improve the selection in choosing
storage for blocks
[ https://issues.apache.org/jira/browse/HDFS-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brahma Reddy Battula updated HDFS-11464:
----------------------------------------
Target Version/s: 3.4.0 (was: 3.3.0)
Bulk update: moved all 3.3.0 non-blocker issues, please move back if it is a blocker.
> Improve the selection in choosing storage for blocks
> ----------------------------------------------------
>
> Key: HDFS-11464
> URL: https://issues.apache.org/jira/browse/HDFS-11464
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Priority: Major
> Attachments: HDFS-11464.001.patch, HDFS-11464.002.patch, HDFS-11464.003.patch, HDFS-11464.004.patch, HDFS-11464.005.patch
>
>
> Currently the logic in choosing storage for blocks is not a good way. It always uses the first valid storage of a given StorageType ({{see DataNodeDescriptor#chooseStorage4Block}}). This should not be a good selection. That means blcoks will always be written to the same volume (first volume) and other valid volumes have no choices. This problem is brought up by this comment ( https://issues.apache.org/jira/browse/HDFS-9807?focusedCommentId=15878382&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15878382 )
> There is one solution from me:
> * First, based on existing storages in one node, extract all the valid storages into a collection.
> * Then, disrupt the order of these vaild storages, get a new collection.
> * Finally, get the first storage from the new storages collection.
> These steps will be executed in {{DataNodeDescriptor#chooseStorage4Block}} and replace current logic. I think this improvement can be done as a subtask under HDFS-11419. Any further comments are welcomed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org