You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Yuanbo Liu (JIRA)" <ji...@apache.org> on 2017/01/06 10:20:58 UTC
[jira] [Comment Edited] (HDFS-11150) [SPS]: Provide persistence
when satisfying storage policy.
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804220#comment-15804220 ]
Yuanbo Liu edited comment on HDFS-11150 at 1/6/17 10:20 AM:
------------------------------------------------------------
uploaded v5 patch for this JIRA.
[~umamaheswararao] / [~rakeshr] please help me review it if you have time. Thanks in advance.
[~umamaheswararao] Due to some scenarios make my restart testing unstable, I've changed some code of {{StoragePolicySatisfier}}, the basic idea is here:
{code}
expected type = [SSD, DISK, DISK]
current block types = [DISK(datanode A), DISK(datanode B), DISK(datanode C)], their storage type is disk, and they all can be added into candidate source list.
||
expected type = [SSD]
source pairs = [DISK => datanode A, DISK => datanode B, DISK => datanode C]
A, B, C are put into exclude node list.
||
if chooseTargetTypeInSameNode = true, then expected type =[], otherwise keep searching by chooseTarget
||
if the expected type is empty, that means all unsatisfied blocks find target nodes.
{code}
was (Author: yuanbo):
uploaded v5 patch for this JIRA.
[~umamaheswararao] / [~rakeshr] please help me review it if you have time. Thanks in advance.
[~umamaheswararao] Due to some scenarios make my restart testing unstable, I've changed some code of {{StoragePolicySatisfier}}, the basic idea is here:
{code}
expected type = [SSD, DISK, DISK]
current block types = [DISK(datanode A), DISK(datanode B), DISK(datanode C)]
||
expected type = [SSD]
source pairs = [DISK => datanode A, DISK => datanode B, DISK => datanode C]
A, B, C are put into exclude node list.
||
if chooseTargetTypeInSameNode = true, then expected type =[], otherwise keep searching by chooseTarget
||
if the expected type is empty, that means all unsatisfied blocks find target nodes.
{code}
> [SPS]: Provide persistence when satisfying storage policy.
> ----------------------------------------------------------
>
> Key: HDFS-11150
> URL: https://issues.apache.org/jira/browse/HDFS-11150
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Reporter: Yuanbo Liu
> Assignee: Yuanbo Liu
> Attachments: HDFS-11150-HDFS-10285.001.patch, HDFS-11150-HDFS-10285.002.patch, HDFS-11150-HDFS-10285.003.patch, HDFS-11150-HDFS-10285.004.patch, HDFS-11150-HDFS-10285.005.patch, editsStored, editsStored.xml
>
>
> Provide persistence for SPS in case that Hadoop cluster crashes by accident. Basically we need to change EditLog and FsImage here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org