You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Marton Elek (Jira)" <ji...@apache.org> on 2020/03/12 11:10:00 UTC
[jira] [Comment Edited] (HDDS-3167) Support different replication
factor for CLOSED Container
[ https://issues.apache.org/jira/browse/HDDS-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057818#comment-17057818 ]
Marton Elek edited comment on HDDS-3167 at 3/12/20, 11:09 AM:
--------------------------------------------------------------
Thanks to open this issue [~Sammi]. I totally agree that it would be useful. But I have a few questions (and long-term ideas).
How would you like to decide the replication factor for a specific container? Do you suggest to use one global settings or some kind of per-container settings?
As a first step a generic settings seems to be enough, but what if somebody change the settings and restart the SCM. Should it be applied for all the remaining containers?
If we would like to support EC in the future it would be great to support per-container replication settings. In fact, we need pairs of settings. For example:
RATIS/THREE (when it's open) --> THREE (when it's closed)
RATIS/ONE (when it's open) --> EC-5-2 (when it's closed)
One approach to support this is introducing a the "storage class", similar to the S3 storage class.
For example
REDUCED = RATIS/ONE (for open) and THREE (for closed))
STANDARD = RATIS/THREE (open) and TWO (closed)
INFREQUENT = RATIS/THREE (open) and EC (closed)
It's not strictly about this issue, just thinking loudly here.
was (Author: elek):
How would you like to decide the replication factor for a specific container? Do you suggest to use one global settings or some kind of per-container settings?
As a first step a generic settings seems to be enough, but what if somebody change the settings and restart the SCM. Should it be applied for all the remaining containers?
If we would like to support EC in the future it would be great to support per-container replication settings. In fact, we need pairs of settings. For example:
RATIS/THREE (when it's open) --> THREE (when it's closed)
RATIS/ONE (when it's open) --> EC-5-2 (when it's closed)
One approach to support this is introducing a the "storage class", similar to the S3 storage class.
For example
REDUCED = RATIS/ONE (for open) and THREE (for closed))
STANDARD = RATIS/THREE (open) and TWO (closed)
INFREQUENT = RATIS/THREE (open) and EC (closed)
It's not strictly about this issue, just thinking loudly here.
> Support different replication factor for CLOSED Container
> ---------------------------------------------------------
>
> Key: HDDS-3167
> URL: https://issues.apache.org/jira/browse/HDDS-3167
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Sammi Chen
> Priority: Major
>
> For open container, it's replication is decided by Ratis pipeline number number. For closed container, this is no such constrain. We can support more replication factor cases, such as 2 factor, to save storage for some cold data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org